Two dimensional barcode with dynamic environmental data system, method, and apparatus

ABSTRACT

Methods, systems, and apparatus for combining preprinted information together with coded sensor information within a two-dimensional barcode. The sensor information may be of an environmental, physical or biological nature, and records a cumulative change in status of the environmental or biological condition to which the labeled product has been exposed. A sensor dye chemistry is employed that undergoes a continuous chemical or physical state change in response to the occurrence of the environmental condition. The continuous change is between an initial state and an end state causing a change in the color state of the sensor dye embedded within the sensor-augmented two-dimensional barcode, encoding sensor digital information. Sensor information is recovered utilizing the error-correction feature during barcode decoding.

CROSS REFERENCES TO RELATED APPLICATIONS

The present application is a continuation-in-part of U.S. applicationSer. No. 15/084,987, filed on Mar. 30, 2016, which is incorporatedherein by reference in its entirety and which claims the benefit of U.S.Provisional Application 62/140,258, filed on Mar. 30, 2015, the entirecontents of each of which are hereby incorporated herein by reference.

BACKGROUND

A barcode is an optical machine-readable representation of data. Atwo-dimensional (2D) barcode (e.g., Data Matrix or QR Code), is atwo-dimensional way to represent information in a bar code. Black andwhite 2D barcodes can represent more data per unit area thanone-dimensional (i.e., linear) barcodes (e.g., Code 39 or Code 128).Barcodes have many uses, including documenting inventory, trackingdeliveries, matching a product against a pricing file, and providinginformation to the user. For some systems, data recovery from barcodesmay be system critical. Many 2D barcode technologies provide robusterror correction capabilities, and typically, using multiple linkedbarcodes or larger sized barcodes may increase data recoverycapabilities. However, space available for barcodes may be limited inmany aspects. Moreover, the current barcode technology may be improvedupon as presently disclosed.

SUMMARY

The present disclosure provides a new and innovative system, methods andapparatus for providing and reading 2D barcodes that include dynamicenvironmental data, where modules of the barcode may continuously (ascontrasted with step-wise) change color state in response toenvironmental conditions. In an exemplary aspect of the presentdisclosure, a sensor-augmented two-dimensional barcode includes asubstrate, a first layer provided on the substrate, and a second layerprovided on the substrate. The first layer includes a sensor dye region,which includes a sensor dye having a chemistry that is configured,responsive to the occurrence of an environmental condition, to undergo acontinuous chemical or physical state change between an initial stateand an end state, causing a change in the color state of the sensor dye.The color state indicates exposure to the environmental condition. Thesecond layer includes a two-dimensional error-correcting barcode symbol,which includes a plurality of modules in a permanent color state. Themodules are optionally square, rectangular, or circular.

In accordance with another exemplary aspect of the present disclosure,an article of manufacture includes pharmaceutical, biological, foodproduct, chemotherapeutic or vaccine; a container holding saidpharmaceutical, biological, food product, chemotherapeutic or vaccineproduct; and a sensor-augmented two-dimensional barcode symbol providedon or in the container, preferably being applied to the outside surfaceof the container.

In accordance with another exemplary aspect of the present disclosure, amethod of reading of a sensor-augmented two-dimensional barcode symbolincludes optically scanning an image of the sensor-augmentedtwo-dimensional barcode symbol to obtain color values for pixels in theimage. Then, a scanned pixel map containing the color values in thesensor-augmented two-dimensional barcode symbol is constructed, and thepixels in the scanned pixel map are processed to assign a binary colorvalue to each pixel and to form a binarised pixel map. Thetwo-dimensional barcode symbol is identified and decoded in thebinarised pixel map to recover a symbol codeword sequence. Next,underlying data codewords are recovered from the symbol codewordsequence, preferably by utilizing error correction process on the symbolcodeword sequence. The data codewords are processed for identificationof barcode modules in a sensor dye region, and an average color value ofthe barcode modules in the sensor dye region is determined. The averagecolor value of the sensor dye region is processed to determine areflectance percentage of incident light at a time of scanning.

In accordance with another exemplary aspect of the present disclosure, amethod of reading of a sensor-augmented two-dimensional barcode symbolincludes a sensor-augmented two-dimensional barcode symbol includesoptically scanning an image of the sensor-augmented two-dimensionalbarcode symbol to obtain a greyscale value for each pixel in the image.Then, a greyscale pixel map of the pixels in the sensor-augmentedtwo-dimensional barcode symbol is constructed. Next, the method includesprocessing the pixels in the greyscale pixel map to assign a binarycolor value to each pixel and to form a binarised pixel map.Additionally, identifying the two-dimensional barcode symbol in thebinarised pixel map is identified and decoded to recover a symbolcodeword sequence. Then, underlying data codewords from the symbolcodeword sequence are recovered utilizing an error-correction process onthe symbol codewords. The data codewords are processed foridentification of the barcode modules in a sensor dye region. Then, anaverage greyscale value of the barcode modules in the senor dye regionis determined, and the average greyscale values of the sensor dye regionare processed to determine a reflectance percentage of incident light ata time of scanning.

Additional features and advantages of the disclosed system, method, andapparatus are described in, and will be apparent from, the followingDetailed Description and the Figures.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1A is a block diagram of an example data structure of barcode dataof a 2D barcode, according to an example embodiment of the presentdisclosure.

FIG. 1B is a block diagram of an example data structure of the encodeddata of a 2D barcode, according to an example embodiment of the presentdisclosure.

FIG. 1C is a block diagram of an example data structure of the contentdata of a 2D barcode, according to an example embodiment of the presentdisclosure.

FIG. 1D is a block diagram of an example data structure of the referencedata of a 2D barcode, according to an example embodiment of the presentdisclosure.

FIG. 1E is a block diagram of an example data structure of the payloaddata of a 2D barcode, according to an example embodiment of the presentdisclosure.

FIG. 2A is an example 2D barcode, according to an example embodiment ofthe present disclosure.

FIG. 2B is an example of the reference data of a 2D barcode, accordingto an example embodiment of the present disclosure.

FIG. 2C is an example 2D barcode, according to an example embodiment ofthe present disclosure.

FIG. 2D is an example of the reference data of a 2D barcode, accordingto an example embodiment of the present disclosure.

FIG. 3A is a representation of a 2D barcode, according to an exampleembodiment of the present disclosure.

FIG. 3B is a representation of a 2D barcode, according to an exampleembodiment of the present disclosure.

FIG. 3C is a representation of a 2D barcode, according to an exampleembodiment of the present disclosure.

FIG. 3D is a representation of a 2D barcode, according to an exampleembodiment of the present disclosure.

FIG. 4 includes a flowchart illustrating an example process forproviding a 2D barcode, according to an example embodiment of thepresent disclosure.

FIG. 5 is a flow diagram illustrating a 2D barcode printed using anexample process for providing a 2D barcode, according to an exampleembodiment of the present disclosure.

FIG. 6A is a textual representation of a set of static data and a set ofdynamic data, according to an example embodiment of the presentdisclosure.

FIG. 6B is an example of textual dynamic data encoded as binaryinformation modules, according to an example embodiment of the presentdisclosure.

FIG. 6C is a flow chart diagram illustrating a 2D barcode printed usingan example process for providing a 2D barcode, according to an exampleembodiment of the present disclosure.

FIG. 6D is a flow chart diagram illustrating a 2D barcode printed usingan example process for providing a 2D barcode, according to an exampleembodiment of the present disclosure.

FIG. 7 includes a flowchart illustrating an example process forproviding a 2D barcode, according to an example embodiment of thepresent disclosure.

FIG. 8 is a flow diagram illustrating a portion of 2D barcode printedusing an example process for providing a 2D barcode, according to anexample embodiment of the present disclosure.

FIG. 9 includes a flowchart illustrating an example process forproviding a 2D barcode, according to an example embodiment of thepresent disclosure.

FIG. 10A is a block diagram of an example set of information modulesprinted using an example process for providing a 2D barcode, accordingto an example embodiment of the present disclosure.

FIG. 10B is a block diagram of an example set of information modulesprinted using an example process for providing a 2D barcode, accordingto an example embodiment of the present disclosure.

FIG. 10C is a block diagram of an example set of information modulesprinted using an example process for providing a 2D barcode, accordingto an example embodiment of the present disclosure.

FIG. 11 includes a flowchart illustrating an example process forproviding a 2D barcode, according to an example embodiment of thepresent disclosure.

FIG. 12 includes a flowchart illustrating an example process for readinga 2D barcode, according to an example embodiment of the presentdisclosure.

FIG. 13 includes a flowchart illustrating an example process for readinga 2D barcode, according to an example embodiment of the presentdisclosure.

FIG. 14 is a block diagram of a 2D barcode providing system, accordingto an example embodiment of the present disclosure.

FIG. 15A is a block diagram of a 2D barcode reading system, according toan example embodiment of the present disclosure.

FIG. 15B is a block diagram of a 2D barcode reading system, according toan example embodiment of the present disclosure.

FIG. 16 is a block diagram of a 14×14 Data Matrix symbol encoding thedata “1234567890”, according to an example embodiment of the presentdisclosure.

FIG. 17 is an illustration of Codeword and utah placement and bitmapmatrix for a 14×14 Data matrix symbol, according to an exampleembodiment of the present disclosure.

FIG. 18 is a Schematic of the general codeword placement within a 10×10Data Matrix bitmap, according to an example embodiment of the presentdisclosure.

FIG. 19 is a block diagram of Invariant utah placement in all practicalsizes of Data Matrix symbols, according to an example embodiment of thepresent disclosure.

FIG. 20 is a representation of a 14×14 bitmap and utah placement withina 16×16 Data Matrix symbol, according to an example embodiment of thepresent disclosure.

FIG. 21 is a representation of utahs used in the GS1 AI (90) string andplacement of the 15 BCH(15,5,7) encoding bits, according to an exampleembodiment of the present disclosure.

FIG. 22 is a block diagram of a 16×16 Data Matrix using W→X sensor dyechemistry and encoding the GS1 AI (90) string, according to an exampleembodiment of the present disclosure.

FIG. 23 is a Detail of utahs 1-7 in FIG. 7, according to an exampleembodiment of the present disclosure.

FIG. 24 is a block diagram of the 16×16 Data Matrix of FIG. 7 withunactivated, overprinted sensor dye modules, according to an exampleembodiment of the present disclosure.

FIG. 25 is a block diagram of a 16×16 Data Matrix using X→B sensor dyechemistry and encoding the GS1 AI (90) string, according to an exampleembodiment of the present disclosure.

FIG. 26 is a Detail of utahs 1-7 in FIG. 25, according to an exampleembodiment of the present disclosure.

FIG. 27 is a block diagram of the 16×16 Data Matrix of FIG. 25 withunactivated, overprinted sensor dye modules, according to an exampleembodiment of the present disclosure.

FIG. 28 is a Detail of a 16×16 Data Matrix encoding the data of Table 8showing a 4×4 frame area, according to an example embodiment of thepresent disclosure.

FIG. 29 is a block diagram of an Underlying 16×16 Data Matrix encodedusing the data on Table 8, according to an example embodiment of thepresent disclosure.

FIG. 30 is a block diagram of the 16×16 Data Matrix of FIG. 29 with anoverprinted 4×4 white frame, according to an example embodiment of thepresent disclosure.

FIG. 31 is a block diagram of the 16×16 Data Matrix of FIG. 30 showingan overprinted and activated sensor dye patch, according to an exampleembodiment of the present disclosure.

FIG. 32A is a block diagram of a 26×26 Data Matrix with a sensor dyepatch in the Invariant space in a first intermediate state between theinitial state and end state, according to an example embodiment of thepresent disclosure.

FIG. 32B is a block diagram of a 26×26 Data Matrix with a sensor dyepatch in the Invariant space in a second intermediate state, accordingto an example embodiment of the present disclosure.

FIG. 33 is a graph of Reflectance R(t) vs. Equivalent Exposure time (t)of a sensor dye patch, according to an example embodiment of the presentdisclosure.

FIG. 34 is a first representation of a barcode reader display, accordingto an example embodiment of the present disclosure.

FIG. 35 is a second representation of a barcode reader display,according to an example embodiment of the present disclosure.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

Prior approaches have been with one-dimensional (linear or 1D) barcodesthat either become unreadable or change with stimuli. Noteworthy, arethe FreshCode smart barcode labels supplied by Varcode Ltd. See:http://www.varcode.com/portfolio_item/freshcode/).

There have been a plurality of 1D barcode patent applications includingNemet et al., US20140252096 A1. While combining environmentalmeasurements into data value of 1D barcodes has been generallydescribed, the techniques applicable to 1D barcodes do not appear to beapplicable to two-dimensional (2D) barcodes with environmentalmonitoring.

The area taken by 1D barcodes limit their practicality in applicationswere space is critical, e.g., unit of use applications, vials, etc.Two-dimensional barcodes with high density encoding technologies, forexample Data Matrix, could have an encoded area approximately 30 timesor more smaller in practice than a 1D barcode representing the samedata.

Prior applications involving 2D barcodes have not addressedenvironmental monitoring and focus on high density of information storedthat is static and not sensitive to environmental factors such astemperature, time, time-temperature product, freezing, nuclearradiation, toxic chemicals, or the like.

Some solutions have addressed having two sets of data, primary/secondaryor covert/overt data, whereby the second set of information is storedinto the redundant space of the barcode and can be read with aconventional reader or independently using a second reader or otherdecrypting methods. While these solutions including the secondary/covertdata have addressed matters such as security and authenticity, none haveaddressed environmental monitoring where data is dynamic. A pending USPatent Application by Porter et al. US 20130015236 A1 High-valuedocument authentication system and method and references containedtherein describe primary and secondary information sets and theapplication to document authentication.

Other solutions have addressed multiple sets of data,primary/secondary/tertiary etc., whereby different data sets (secondary,tertiary, etc.) are progressively stored over the prior set. Multipleinformation sets are incrementally added, for example, by printing overthe prior set with different color modules and then decrypting the datautilizing readers configured to interpret various color modules. See forexample Simske et al., US20140339312 A1. These solutions areprogressively introducing static data one layer at a time and addressmatters pertaining to tracking, tracing, inspection, quality assuranceand not dynamic environmental data.

Some example embodiments described in this disclosure provide a uniqueway of combining preprinted data together with coded sensor informationin a two-dimensional barcode on a media. The preprinted data and codedsensor information may be combined in a single step, or the coded sensorinformation may be added dynamically to the preprinted data in asecondary step depending on the actual planned sensor usage.

A sensor dye chemistry may be employed where the sensed property is anenvironmental, physical, or biological property. A specified conditionof the sensed property causes activation of a chemical or physical statechange resulting in a change in the color state of the sensor dye. Thechange in the color state results in sensor digital information beingrevealed in the pattern of sensor dye modules within thesensor-augmented two-dimensional barcode. Sensor digital information isrecovered when the barcode is read using an extension of the standardreading and error correction algorithms for the two-dimensional barcodesymbology in use.

In another example, the sensor dye may have a chemistry that isconfigured to undergo a continuous chemical or physical state change.For example, the sensor dye may continuously change between an initialstate and an end state causing a change in the color state of the sensordye. In an example, the sensor dye may have a chemical composition thatcontinuously changes color based on exposure to both time andtemperature (e.g., the current color state of the sensor dye patch is aresult of the cumulative exposure to environmental conditions). Forexample, for a product that spoils rapidly at temperatures above 15° C.,the sensor dye patch color change may indicate a short exposure to sucha temperature even if the product is currently at a temperature wellbelow 15° C. Due to the cumulative nature of the sensor dye chemistry,the sensor dye continuously changes color from an initial state to anend state such that product properties, such as remaining product life,can be obtained at any point in the supply chain.

Examples of environmental sensors include temperature monitors,measuring either cumulative heat exposure or passing beyond a set highor low temperature threshold value(s); time, time-temperature product,nuclear radiation exposure monitors; gas or humidity exposure monitorseach passing above a cumulative exposure threshold or an instantaneousthreshold value. Examples of medical sensors include recording patientthermometers; assays measuring levels to biological toxins such asaflatoxin or botulism toxin; and includes colorimetric immunoassays forsensing of the presence of biological agents such as prions orbiological organisms such as infectious bacteria.

Block diagrams of an example data structure 100 of a two dimensional(2D) barcode, an example data structure of the encoded data of a 2Dbarcode, an example data structure of the content data of a 2D barcode,and an example data structure of the reference data of a 2D barcode areshown in FIG. 1A, FIG. 1B, FIG. 1C, and FIG. 1D respectively. The 2Dbarcode includes encoded data 104 and reference data 106. The encodeddata 104 may include content data 108 and format and version data 110.The content data 108 may include payload data 112, padding data 114,error detection and correction data 116, and remainder space 118. Theformat and version data 110 provides information needed to decode thecontent data 108. For example, the format and version data 110 mayinclude mask information. The reference data 106 is the data required toidentify which protocol is being used and allows a reader to locate andscan the 2D barcode. The reference data 106 may include alignment data124, finder data 126, timing data 128, positioning data 130, andorientation data 132. The payload data may include static data 134 anddynamic data 136. For example, as illustrated in FIGS. 2A, 2B, 2C, and2D discussed in further detail below, the positioning data 130 mayinclude a position block 138 or positioning modules, and the finder data126 may include a finder pattern 142 or finder modules. Additionally,the alignment data 124 may include an alignment block 146. The timingdata 128 may include a timing pattern 148. The reference data 106 mayalso include separators, identification modules, and orientationmodules.

Example 2D barcodes are shown in FIG. 2A and FIG. 2C and theirrespective reference data 106 and format and version data 110 are shownin FIG. 2B and FIG. 2D. FIG. 2A includes all the encoded data 104 andreference data 106 for a 2D barcode 156 (e.g., Data Matrix). FIG. 2Bshows only the reference data 106 for a 2D barcode 156. For example, thereference data 106, which includes the finder pattern 142 and the timingpattern 148, encloses the content data 108 of the 2D barcode 156. FIG.2C includes all the encoded data 104 and reference data 106 for a 2Dbarcode 158 (e.g., QR code). FIG. 2D shows only the reference data 106for a 2D barcode 158. For example, the reference data 106 may includethe position blocks 138, the alignment block 146, and the timingpatterns 148.

A representation of a 2D barcode 156 is shown in FIG. 3A and FIG. 3B. Arepresentation of a 2D barcode 158 is shown in FIG. 3C and FIG. 3D. Therepresentation of each 2D barcode shows the reference data region 160and the encoded data region 162. The encoded data region 162 may includestatic data 134, dynamic data 136, and error detection and correctiondata 116. Also, the 2D barcode 156 may include a plurality of unusedbits which may not be used for encoded data 104 or providing referencedata 106. The 2D barcode 158 may include remainder space 118 which mayinclude a plurality of unused bits.

Some examples described herein employ Data Matrix, although it will beappreciated that similar approaches may be employed with other twodimensional bar codes schemes by varying the approach to conform withthe applicable 2D barcode standard. Data Matrix is a two-dimensionalerror-correcting barcode symbol formed in compliance with ISO/IEC 16022Information technology—automatic identification and data capturetechniques Data Matrix barcode symbology specification. The ECC 200 DataMatrix symbology utilizes Reed-Solomon error correction to recover theencoded data from symbols which have suffered a limited amount ofaccidental damage or deliberate alternation. All Data Matrix referred toherein are of the ECC 200 symbol type, and may be either square orrectangular in shape, and are identified by the number of rows andcolumns respectively.

Data is encoded in a Data Matrix as a sequence of 8-bit codewords, orsymbol character values. Codewords may either contain data orReed-Solomon error correction (RSEC) check character values. It will beappreciated that the general approach described herein may use othercodeword sizes, other data layouts, and other forms of error correctingcodes, and that this common Data Matrix is only described as an example.

Each module is a visual cell in matrix comprising the Data Matrix symbolthat is used to encode one bit of data. Each module is colored eithernominally black or nominally white. The module matrix is the visualmanifestation of the binary bitmap matrix contained with the area of thesymbol bounded by the Finder Pattern. The Finder Pattern may be an ‘L’formed by connected solid lines along two edges of the symbol modulematrix, with an alternating pattern of white and black modules along theopposite edges of the symbol. See FIG. 16. It will be appreciated thatin other bar code symbologies, other finder patterns may be employed.

FIG. 16 illustrates a 14×14 square Data Matrix 320 encoding the data“1234567890”. This Data Matrix has two parts: the Finder Pattern formingan ‘L’ formed by connected solid lines 312 a and 312 b along two edgesof the symbol with an alternating pattern of white and black modules 322a and 322 b along the opposite edges of the symbol. Symbol codewords areencoded in the 12×12 module matrix 330 interior to the Finder Pattern.

The detailed structure of the 12×12 module matrix 330 of the 14×14 DataMatrix 320 is shown as the bitmap matrix 200 in FIG. 17. A 14×14 DataMatrix contains 18 codewords, each formed of eight modules correspondingto the eight bits of the codeword, referred to as a “utah.” The 12×12bitmap matrix shows the layout of all the 18 codewords in a 14×14 DataMatrix.

A “utah” is an arrangement of 8 modules to encode 1 codeword. It may bearranged either as a single connected group with a pattern frequently inthe shape of the State of Utah, or formed as two subgroups of connectedmodules split across two patterns. The contiguous utah 270 encodingcodeword 9 shows the typical arrangement of bits within a contiguousutah. Conversely, the utah for codeword 4 consists of two smallersubgroups: Subgroup 272 a at the top of the bitmap matrix 200 encodingbits 4.3 through 4.8, and subgroup 272 b at the bottom of the bitmapmatrix encoding bits 4.1 and 4.2.

The general layout of codeword utah placement within a 10×10 Data Matrixbitmap matrix 300 is shown in FIG. 18. Trace lines 310 shows the overallmethod for placing codeword utahs within a bitmap matrix. Note incomparing FIGS. 17 and FIG. 18 that the placement of:

All bits of utah 2

Utah 3 bits 3.6-3.8

Utah 4 bits 4.3-4.8

All bits of utahs 5 and 6

These bit positions in identical positions relative to the upperleft-hand corner (ULC) of the Data Matrix symbol. In accordance withISO/IEC 16022 standard Annex F.3, all square Data Matrix symbols up tosize 26×26 and all rectangular Data Matrix symbols, these bit positionsare invariant in their placement relative to the ULC of each Data Matrixsymbol. These bit positions define an “invariant bitmap” for Data Matrixsymbols. It will be appreciated that other barcode standards may havedifferent invariant bitmaps. In FIG. 19, invariant bitmap 410 is shownin the ULC of Data Matrix symbol 405. To define terminology, the printedData Matrix symbol prior to augmentation with sensor modules is referredto as the “underlying Data Matrix symbol”; its codeword sequence as the“underlying symbol codeword sequence” encoding “underlying datacodewords and their RSEC error correction codewords”. It will beappreciated that other symbologies have their own underlying symbols,underlying codeword sequences, underlying data codewords and errorcorrection codewords, depending on the particular type of errorcorrection employed.

FIG. 4 includes a flowchart of an example process 400 of providing a 2Dbarcode. Although the process 400 is described with reference to theflowchart illustrated in FIG. 4, it will be appreciated that many othermethods of performing the acts associated with the process 400 may beused. For example, the order of many of the blocks may be changed, manyblocks may be intermittently repeated or continually performed, certainblocks may be combined with other blocks, and many of the blocksdescribed are optional or may only be contingently performed.

The example process 400 may begin with determining a set of payload data112 which includes a set of static data 134 and a set of dynamic data136 (block 402). For example, the set of static data 134 may include aserial number, a lot number, a batch number, and a threshold exposuretemperature for a product (e.g., 30° C.). Additionally, the set ofdynamic data 136 may include information that when decoded by a readerinforms a user whether the product exceeded the threshold exposuretemperature. Next, a 2D barcode which includes an encoded version of theset of static data is generated, and the 2D barcode includes redundantspace (block 404). In an example embodiment, the redundant space mayinclude a plurality of unused bits, a padding region, and/or errordetection and correction region (e.g., remainder space 118).Additionally, the redundant space may include a format informationregion, a version information region, and/or a reference data region.Furthermore, the redundant space may include all of the remainder spaceand/or all of the plurality of unused bits. Additionally, the redundantspace may include a portion of the padding data, or may include all ofthe padding data. The redundant space may also include a portion of thecontent data, a portion of the reference data, or a combination ofvarious portions and/or all of data included in the content data and thereference data. Next, at least part of the redundant space is designatedas a dynamic region which is adapted to store the set of dynamic data136 (block 406). Next, the 2D barcode is printed using a static ink andan encoded version of the set of the dynamic data on the dynamic regionusing a dynamic ink which changes states responsive to at least oneenvironmental change, such that the set of dynamic data 136 is in one ofa plurality of states (block 408). In an example embodiment, the set ofdynamic data 136 is readable by a reader of the 2D barcode and the setof static data 134 is readable by the reader of the 2D barcode when theset of dynamic data 136 is in each one of the plurality of states. In anexample embodiment, the dynamic ink may be sensitive to an environmentalfactor such as temperature, time, time and temperature, freezing,radiation, toxic chemicals, or a combination of such factors, or thelike. In an example embodiment, the ink may be a thermochromic ink. Forexample, the dynamic ink may be a water-based irreversible thermochromicink designed to change permanently from white to black at 40° C.Additionally, the thermochromic ink may be reversible. For example, thereversible thermochromic ink may be a liquid crystal ink or a leuco dyeink (examples include QCR Solutions Reversible Thermochromic Inks andH.W. Sands Corporation inks). In an example embodiment, the ink may be aphotochromic ink which may be either reversible or irreversible. Forexample, the dynamic ink may change states based upon exposure to UVlight. Additionally, the ink may be an ink sensitive to time andtemperature (an example includes the OnVu indicator). The dynamic inkmay change from a darker color to a lighter color, a lighter color to adarker color, may change levels of transparency or opacity, and/or maychange levels of reflectivity or absorptivity, or may change any othersuitable characteristic allowing the barcode to be readable in one ormore states by a reader. Additionally, the dynamic ink may continuouslychange between a range of an initial color state to an end color state.For example, a dynamic ink may change from a lighter color to a darkblue, which may be alternatively transformed by a reader to values oncontinuous “greyscale”. Said greyscale (which is not necessary trulygrey but is a continuous tone of some hue) is determined reducing the R,G and B values of each pixel to a single greyscale value by a formula ofform:

Greyscale value=(aR+bG+cB)/K

Where {a, b, c} represent the relative contribution of each sRGB colorin the pixel, and K is scaling factor. Additionally, a dynamic ink orsensor dye patch may continuously change from a white or clear color toa dark red or blue (e.g., changing from white, to a faint red, becomeless and less opaque until it reaches a solid red color at the end colorstate). Moreover, any suitable combination of colors may be used for thestates of one or more dynamic inks.

In an example embodiment, the dynamic ink may permanently change orirreversibly change in response to an environmental factor. For example,a certain drug may experience chemical decomposition when it is exposedto a certain temperature. A supplier may want to know if the drug everreached a temperature above 30° C. threshold during transportation andstorage. If the dynamic ink irreversibly changes, the set of dynamicdata 136 may be decoded to inform the supplier that the chemical wasexposed to a temperature above 30° C. and that the contents of thecontainer may have experienced thermal degradation. Providing a 2Dbarcode with the set of dynamic data 136 in the redundant spaceadvantageously allows an individual to use a reader to read the 2Dbarcode both before the dynamic ink has activated and after the dynamicink has activated from the set of dynamic data 136 in a first state torepresent a set of dynamic data 136 in a second state. For example, theoriginal 2D barcode is still readable and allows a reader to obtain theset of static data 134, such as serial number, lot number, and batchnumber even after the set of dynamic data 136 is in a second state, thisadvantageously allows both the set of static data 134 and the set ofdynamic data 136 to be printed on the same 2D barcode without losing thestatic data 134 upon the set of dynamic data 136 changing from the firststate to the second state.

In another example embodiment, the dynamic ink may continuously changein response to an environmental factor. For example, the sensor dye maychange colors responsive to the occurrence of an environmentalcondition, such as time, temperature, or time-temperature product. Forexample, it may be advantageous for a product user to know the remainingproduct life instead of just whether or not the product has alreadyexpired. For example, products closer to expiration may have theirproduct life extended if kept at a lower condition or by reducingexposure to a specific environmental condition. Additionally,consumption or use of products that are near expiration may beprioritized, thereby advantageously reducing waste. By having a sensordye that continuously changes color, product life can be determinedthroughout the products lifetime and throughout the supply chain.

A sensor dye chemistry is utilized to detect the change in condition ofa sensed environmental or medical property. Since Data Matrix modulesare nominally either black or white, 6 types of sensor dye chemistriesmay be utilized as shown in Table 1. Here “B” refers to black colorstate; “W” refers to white color state; and “X” refers to the dye beingin a transparent color state.

TABLE 1 Types and Color Properties of Selected Sensor Dye ChemistriesSensor dye chemistry Unactivated State Activated State W→X WhiteTransparent W→B White Black B→W Black White B→X Black Transparent X→WTransparent White X→B Transparent Black

It is assumed that black and white dye color states are opaque and thushide the underlying color when in either the black or white state.However, when a dye is in the transparent color state (X), theunderlying color is now visible.

Since sensor dye module patterns can be either overprinted on eitherblack or white modules of the Data Matrix symbol, or printed in place ofData Matrix modules, the color state of the sensor dye modules in theData Matrix symbol changes upon sensor dye chemistry activation.Different sensor dye chemistry systems have different properties withrespect to encoding sensor data.

The sensor dye systems which go from the colored to transparent stateare typically used with an underlying bit pattern that is printed in theData Matrix, and all of the bits in that pattern are covered with sensordye creating a uniform black or white color state covering those modulesuntil activated. Once activated, the sensor dye turns transparent andthe underlying bit pattern in the Data Matrix barcode is visible.Alternately dye systems which go from transparent color state to a blackor white color state may selectively cover modules of a uniform black orwhite pattern printed in the Data Matrix symbol, and upon activation,the data encoded in the sensor dye module pattern is now visible. Thislatter system has the advantage that the sensor dye encoded data may bedetermined at the time of printing of the sensor dye modules which maybe at a different time and location than the pre-printing of the DataMatrix symbol.

In the example, the sensor dye chemistry in use can be accuratelyprinted on a module by module basis over a preprinted Data Matrixmodule, or printed in place of Data Matrix module. For example, this maybe done on a multi-station printing press. Another method of doing thisis on-demand through the use of a 2-channel piezoelectric inkjetprinter, where one channel contains black ink for printing the DataMatrix and the other channel contains sensor dye to be printed either ontop of or in place of Data Matrix modules. The final Data Matrix symbol,augmented by the addition of sensor modules, is one example of a“sensor-augmented two-dimensional barcode symbol”.

If a 2-channel inkjet printer is used, then the sensor dye chemistry inuse should be known at the time of printing, especially if sensormodules are printed in place of normally-printed Data Matrix symbolmodules.

In the case of overprinted sensor modules where the sensor dye chemistryis known at the time of printing the underlying Data Matrix, it ispossible to encode indicator bits itself which specify the dye system inuse. Indicator bits in the underlying Data Matrix are over printed withsensor dye modules. Use of these indicator bits will enable the DataMatrix reader to know what dye system is in use and its activation stateand thus how to interpret a scanned Data Matrix symbol.

Note that in Table 2, 0=white and 1=black for each indicator bit. Onlythe 4 most common dye chemistries, where one of the states istransparent and the other is either black or white, are coded by theindicator bits.

TABLE 2 Indicator Bits for the Most Common Sensor Dye Chemistries SensorDye Data Matrix Printed Indication Before Indication After System Bits3.6 and 3.7 Sensor Activation Sensor Activation W→X 11 00 11 B→X 00 1100 X→B 10 10 11 X→N 01 01 00

The value of the printed indicator bits may be recovered from thescanned image regardless of whether it is before or after sensoractivation, and may be compared with the indicator bits read from theunderlying Data Matrix. Then both the dye system and activation stateare determined according to Table 3.

TABLE 3 Recovering the Activation State and Indicator Bits from theScanned Image Underlying Data Scanned Sensor Dye Sensor Dye MatrixIndicator Bits Indicator Bits System State 00 00 B→X Activated 00 11 B→XUnactivated 01 00 X→W Activated 01 01 X→W Unactivated 10 10 X→BUnactivated 10 11 X→B Activated 11 00 W→X Unactivated 11 11 X→WActivated

The colors values of sensor dye modules themselves may be used to encodesensor data which either only becomes visible or changes due to modulecolor state change upon sensor dye activation.

encoding this sensor data itself in an error correcting code is usefulwhen the sensor color change activation threshold is not precise foreach sensor dye module; where the entire sensor-augmentedtwo-dimensional barcode may not have been exposed uniformly to thecondition activating each sensor dye module; or when sensor modules maybe missing or damaged.

Here it is assumed that the sensor module colors may be binarised to 0or 1. For, example 5 bits of sensor module color data may encode twouseful pieces of data: The sensor product type and the activationcondition. The associated parameters may encoded using internal tablesindexed by the 5-bit data value.

Many types of error correcting codes may be used to encode sensordigital information. Typically encoded is a sensor dye bit pattern ofbinary-encoded sensor data. Useful error correcting codes includeHamming Codes, Bose-Chaudhuri-Hocquenghem Codes, Golay Codes, SimplexCodes, Reed-Muller Codes, Fire Codes, Convolutional Codes, andReed-Solomon Codes.

As an example, Bose-Chaudhuri-Hocquenghem BCH (n,k,t) binary errorcorrecting codes are well known for encoding a string of binary data;here a string of k bits are within an n-bit length code with T errorcorrection bits. Up to (t div 2) bits may be error corrected. Forexample, the QR Code and Ultracode 2-dimensional barcode symbologies usea BCH(15,5,7) code which can encode k=5 bits in n=15 bits and correct upto (7 div 2)=3 bit errors. There are standard decoding and errorcorrection techniques for decoding and error correcting these codes.Table C.1 in Annex C of ISO/IEC 18004 Information technology—Automaticidentification and data capture techniques QR Code 2005 barcodesymbology specification gives the complete 15 bit code sequences fordata values 0, 1 . . . 31. ISO/IEC 18004 Table C.1 is summarized inTable 4. Data Values 0 and 31 are reserved and not to be encoded, asthey represent unactivated dye module states: Data value 0 (all BCHbits=0) for W→X and W→B Sensor dye chemistries; data value 31 (all BCHbits=1) for B→X and B→W sensor dye chemistries.

TABLE 4 BCH(15, 5, 3) Encoding of 5 Bits of Sensor Data 5-Bit Data BCH(15, 5, 3) Encoding Value Bit 15 Bits 14 -> 8 Bits 7 -> 1 0 0 00000000000000 1 0 0001010 0110111 2 0 0010100 1101110 3 0 0011110 1011001 4 00100011 1101011 5 0 0101001 1011100 6 0 0110111 0000101 7 0 01111010110010 8 0 1000111 1010110 9 0 1001101 1100001 10 0 1010011 0111000 110 1011001 0001111 12 0 1101110 0111101 13 0 1101110 0001010 14 0 11100001010011 15 0 1111010 1100100 16 1 0000101 0011011 17 1 0001111 010110018 1 0010001 1110101 19 1 0011011 1110101 20 1 0100110 1000000 21 10101100 1000111 22 1 0110010 0011110 23 1 0111000 0101001 24 1 10000101001101 25 1 1001000 1111010 26 1 1010110 0100011 27 1 1011100 001010028 1 1100001 0100110 29 1 1101001 0010001 30 1 1110101 1001000 31 11111111 1111111

Recovery of the sensor bit data when BCH(15,5,7) is encoded in a DataMatrix symbol (whether the sensor bits have been activated or not) maybe done by first recovering the sensor digital information from scanningand decoding on the sensor-augmented two-dimensional barcode symbol.From the particular sensor dye module pattern, extract the 15-bitBCH-encoded binary number. Note that there are many standard methods fordecoding and error-correcting BCH encoded data. The classicPeterson-Gorenstein Zierler Decoder is discussed in R. E. Blahut,“Theory and Practice of Error Control Codes (corr. edition)”, 1983(ISBN-10: 0-201-10102-5), p. 166. A useful and practical algorithm for aBCH(15,5,7) decoding is given by S. A Vanstone and P. C. van Oorschot,“An Introduction to Error Correcting Codes with Applications”, 1989(ISBN-10: 0-7923-9017-2), p. 219. Using any method for decoding anderror-correcting BCH(15,5,7) encoded data, extract the 5-bit sensordata.

Other types of sensor digital information may be encoded in the sensordye module pattern. This includes visual patterns and images of anytype, such as ISO or ANSI or ISO warning signs and symbols, or any othertype of designed graphic. The number of bits which are encoded and thenumber of utahs deliberately damaged in the process (which is affectedby the physical extent of the visual pattern on the underlying DataMatrix symbol) as well as the size of the underlying Data Matrix and itsavailable number of RSEC codewords all affect the visual pattern andimage encoding capability.

Sensor dye module patterns may also be encoded after the underlying DataMatrix symbol has been preprinted. This allows different technologies tobe used for printing the underlying Data Matrix and later printing thesensor dye module pattern. This also allows different kinds of sensordye chemistries to be used on a previously printed Data Matrix withoutthat sensor dye chemistry being known at the time that the underlyingData Matrix itself was printed.

In an example embodiment, a non-privileged reader may be capable ofreading the static data 134 and may not be capable of reading thedynamic data 136 of the 2D barcode. In another example embodiment, onlyprivileged readers may be capable of reading the static data 134 and thedynamic data 136 of the 2D barcode. In some cases, having dynamic data136 on a barcode that cannot be read by non-privileged readers mayadvantageously allow a manufacturer or supplier to include informationon a 2D barcode that they may not want to provide to the public or acustomer. Additionally, having a privileged reader that may read boththe static data 134 and the dynamic data 136 advantageously allows anindividual using the privileged reader to obtain both the static data134 and the dynamic data 136 without having to use multiple readers.

FIG. 5 is a flow chart diagram 500 illustrating an example 2D barcodeprinted using process 400. As shown in FIG. 5, the 2D barcode is printedby a process 502 with an encoded version of the set of dynamic data onthe dynamic region 192. In this example, the set of dynamic data 136 isprinted on a plurality of unused bits 120 in the lower right hand cornerof the 2D barcode. The dynamic ink 198 is used on the upper left handand bottom right hand information modules of the plurality of unusedbits 120. After the dynamic region is printed with the dynamic ink 198,the barcode is in a first state 504 (i.e., the dynamic ink 198 has notactivated yet). The dynamic ink 198 used in the dynamic region 192activates to a solid black color upon exposure to a specifiedenvironmental change 506. Once the 2D barcode is exposed to theenvironmental change 506, the set of dynamic data 136 in the dynamicregion 192 changes and the 2D barcode changes to a second state 508 andconveys the information of the environmental change 506 to a reader.

FIG. 6A is a textual representation of a set of static data 134 and aset of dynamic data 136 that may be encoded into a 2D barcode, and FIG.6B is an example of textual dynamic data encoded as binary informationmodules. FIG. 6C and FIG. 6D are flow chart diagrams illustrating thedynamic data of a 2D barcode changing from a first state to a secondstate. For example, the set of static data 134 includes productinformation such as serial number (SN), batch number (BN), and lotnumber (LN). In this example, the serial number is 498760003, the batchnumber is 654, and the lot number is 35A1. The set of static data 134does not change. Additionally, the set of dynamic data 136 is shown in afirst state 210 (top) and a second state 212 (bottom) in FIG. 6A. Forexample, the set of dynamic data 136 may be printed with a dynamic ink198 that changes from a first state 210 to a second state 212 uponreaching a threshold exposure temperature above 30° C. The textualrepresentation of the set of dynamic data 136 in the first state 210 is“<” (i.e., less than) the threshold exposure temperature specified inthe set of static data 134, and the textual representation of the set ofdynamic data 136 in the second state 212 is “>” (i.e., greater than) thethreshold exposure temperature specified in the set of static data 134,in this case 30° C. In an example embodiment, the 2D barcode may beencoded using a binary 8-bit representation of the set of static data134 and the set of dynamic data 136, as shown in FIG. 6B. For example,the binary representation of “<” may be “00111100” and the binaryrepresentation of “>” may be “00111110.” In the present example, binaryzero (0) bits are colored white and the binary one (1) bits are coloredblack. It should be appreciated that various other color combinationsmay be used to print the 2D barcode, and various other encodingmethodologies may be used. The colors of black and white and binary(8-bit) encoding have been provided for illustrative purposes. FIG. 6Crepresents a 2D barcode with the set of dynamic data 136 in a firststate 210 transitioning to a second state 212 in response to anenvironmental change such as the temperature rising above 30° C. Forexample, the dynamic ink 198 used in the 2D barcode represented in FIG.6C shows a dynamic ink 198 that activates from white to black inresponse to an a temperature greater than the threshold exposuretemperature. In another example embodiment, shown in FIG. 6D, thedynamic ink 198 may activate from black to white in response to anenvironmental change 506, such as freezing or exposure to a thresholdexposure temperature below 0° C.

FIG. 7 includes a flowchart of an example process 420 of providing a 2Dbarcode as illustrated in FIG. 8, discussed in further detail below.Although the process 420 is described with reference to the flowchartillustrated in FIG. 7, it will be appreciated that many other methods ofperforming the acts associated with the process 420 may be used. Forexample, the order of many of the blocks may be changed, many blocks maybe intermittently repeated or continually performed, certain blocks maybe combined with other blocks, and many of the blocks described areoptional or may only be contingently performed.

The example process 420 may begin with determining a set of static data134 (block 422). For example, the set of static data 134 may be a serialnumber, a batch number, and/or a lot number, etc. Next, a set of dynamicdata 136 is determined (block 424). In an example embodiment, the set ofdynamic data 136 may have a first state 210 and a second state 212. Forexample, dynamic data 136 in a first state may be unexposed to UV lightwhile the dynamic data 136 in a second state may be exposed to UV light.Additionally, dynamic data 136 in a first state may be a temperatureless than 30° C. while the dynamic data 136 in a second state may be atemperature greater than 30° C. Then, a first 2D barcode is generated(block 426). For example, a computer may generate the first 2D barcodebased on inputs of the set of static data 134 and the set of dynamicdata 136 in the first state 210. In an example embodiment, the first 2Dbarcode may include an encoded version of the set of static data and theencoded version of the set of dynamic data 196 in the first state 210.Additionally, the set of static data 134 and the set of dynamic data 136in the first state 210 may include a first plurality of informationmodules 218 and a second plurality of information modules 220. Forexample, the first plurality of information modules 218 may be blackmodules and the second plurality of information modules 220 may be whiteinformation modules. Then, a second 2D barcode is generated (block 428).For example, a computer may generate the second 2D barcode based oninputs of the set of static data 134 and the set of dynamic data 136 inthe second state 212. In an example embodiment, the second 2D barcodemay include an encoded version of the set of static data and an encodedversion of the set of dynamic data 196 in the second state 212.Additionally, the set of static data 134 and the set of dynamic data 136in the second state 212 may include a third plurality of informationmodules 224 and a fourth plurality of information modules 226. Forexample, the third plurality of information modules 224 may be blackmodules and the fourth plurality of information modules 226 may be whiteinformation modules. It should be appreciated that the first pluralityof information modules 218 and the third plurality of informationmodules 224 may be white while the second plurality of informationmodules 220 and the fourth plurality of information modules 226 may beblack. Furthermore, it should be appreciated that the first plurality ofinformation modules 218 and the third plurality of information modules224 and/or the second plurality of information modules 220 and thefourth plurality of information modules 226 may be various colors,levels of transparency, and/or levels of reflectivity, or may have anyother suitable characteristic allowing the 2D barcode to be readable bya reader. In an example embodiment, the third plurality of informationmodules 224 may include all of the first plurality of informationmodules 218 plus a set of one or more information modules 228.Additionally, the second plurality of information modules 220 mayinclude all of the fourth plurality of information modules 226 plus theset of one ore more information modules 228. Then, compare the first 2Dbarcode and the second 2D barcode (block 430). For example, the first 2Dbarcode and the second 2D barcode may include binary data having valuesof zero (0) or one (1), which may correspond to each information modulebeing colored white for a binary value of zero (0) and black for abinary value of one (1). Then, categorize the information modules 214into a first group 230 and a second group 232 (block 432). In an exampleembodiment, the first group 230 may include common information modulesbetween the first plurality of information modules 218 of the first 2Dbarcode and the third plurality of information modules 224 of the second2D barcode. Additionally, the second group 232 may include uniqueinformation modules of the third plurality of information modules 224 ofthe second 2D barcode. For example, a computer may categorize all theblack information modules that are common to the first 2D barcode andthe second 2D barcode into the first group 230. Additionally, thecomputer may categorize all the black information modules that areunique to the second 2D barcode (i.e., the information modules that werewhite in the first 2D barcode and are black in the second 2D barcode)into the second group 232. Then, print the 2D barcode using a static ink194 and a dynamic ink 198 (block 434). In an example embodiment, thefirst group 230 may be printed in the static ink 194, and the secondgroup 232 may be printed in the dynamic ink 198. Additionally, thedynamic ink 198 may be adapted to activate in response to the occurrenceof a specific environmental factor. In an example embodiment, thedynamic ink 198 may be sensitive to an environmental factor such astemperature, time, time and temperature, freezing, radiation, toxicchemicals, or a combination of such factors, or the like.

In an example embodiment, the first plurality of information modules 218and the third plurality of information modules 224 may be adapted to bevisually distinguishable from a printing surface 234 and the thirdplurality of information modules 224 and the fourth plurality ofinformation modules 226 may be visually indistinguishable from theprinting surface 234. It should be appreciated that various printingtechniques may be used that involve printing with ink, dye, paint,and/or any other suitable material, or the like. Additionally, variousother techniques may be used to change the visual appearance of theprinting surface 234 of the 2D barcode such as etching, burning,melting, removing material, and/or any other process adapted to print a2D barcode. For example, a printing surface 234 may include a whitesubstrate covered with a black top layer which is etched away to revealthe white substrate underneath. Additionally, a printing surface 234 mayinclude a blue substrate covered with a yellow top layer, along withvarious other color combinations.

FIG. 8 is flow diagram illustrating an example portion of a 2D barcodeprinted using process 420. As shown in FIG. 8, the first 2D barcodeportion 216 is a portion of an example barcode that includes an encodedversion of a portion of the set of static data and an encoded version ofa portion of the set of dynamic data 196 in a first state 210. Thesecond 2D barcode portion 222 is the portion of the example barcode thatincludes the encoded version of the portion of the set of static dataand the encoded version of the portion of the set of dynamic data 196 ina second state 212. All of the black information modules are the firstgroup 230 of common information modules between the first plurality ofinformation modules 218 of the first 2D barcode portion 216 and thethird plurality of information modules 224 of the second 2D barcodeportion 222. Additionally, the information modules outlined in dottedlines may be the second group 232 of unique information modules of thethird plurality of information modules 224 of the second 2D barcodeportion 222. For example, the second group 232 of information modulesmay include all of the information modules that are changing from whiteto black when the set of dynamic data 136 transitions from the firststate 210 to the second state 212. Only using one dynamic ink 198advantageously allows the 2D barcodes to be printed in a more efficientand cost effective manner. Additionally, the use of only one dynamic ink198 advantageously reduces the risk of the code becoming unreadable dueto an offset or delayed activation time of multiple dynamic inks. Itshould be appreciated that the example embodiments disclosed herein maytranslate to various 2D barcodes including an Aztec Code, Code 1,CrontoSign, CyberCode, DataGlyphs, Data Matrix, Datastrip code, EZcode,High Capacity Color Barcode, InterCode, MaxiCode, MMCC, NexCode, PDF417,Qode, QR code, ShotCode, SPARQCode, and the like.

FIG. 9 includes a flowchart of an example process 440 of generating a 2Dbarcode. Although the process 440 is described with reference to theflowchart illustrated in FIG. 9, it will be appreciated that many othermethods of performing the acts associated with the process 440 may beused. For example, the order of many of the blocks may be changed, manyblocks may be intermittently repeated or continually performed, certainblocks may be combined with other blocks, and many of the blocksdescribed are optional or may only be contingently performed.

The example process 440 may begin with determining a set of static data134 (block 442). For example, the set of static data 134 may be a serialnumber, a batch number, and/or a lot number, etc. Next, a set of dynamicdata 136 is determined (block 444). In an example embodiment, the set ofdynamic data 136 may have a first state 210 and a second state 212. Theset of dynamic data 136 may have more than two states. For example, theset of dynamic data 136 may have a first state 210, a second state 212(e.g., activated by going above 25° C.), and a third state (e.g.,activated by going above 40° C.). Then, a first 2D barcode is generated(block 446). For example, a computer may generate the first 2D barcodebased on inputs of the set of static data 134 and the set of dynamicdata 136 in the first state 210. In an example embodiment, the first 2Dbarcode may include an encoded version of the set of static data and anencoded version of the set of dynamic data 196 in the first state 210.Additionally, the set of static data 134 and the set of dynamic data 136in the first state 210 may include a first plurality of informationmodules 218 and a second plurality of information modules 220. Forexample, the first plurality of information modules 218 may be blackmodules and the second plurality of information modules 220 may be whiteinformation modules. Then, a second 2D barcode is generated (block 448).For example, a computer may generate the second 2D barcode based oninputs of the set of static data 134 and set of dynamic data 136 in thesecond state 212. In an example embodiment, the second 2D barcode mayinclude an encoded version of the set of static data and an encodedversion of the set of dynamic data 196 in the second state 212.Additionally, the set of static data 134 and the set of dynamic data 136in the second state 212 may include a third plurality of informationmodules 224 and a fourth plurality of information modules 226. Forexample, the third plurality of information modules 224 may be blackmodules and the fourth plurality of information modules 226 may be whiteinformation modules. It should be appreciated that the first pluralityof information modules 218 and the third plurality of informationmodules 224 may be white while the second plurality of informationmodules 220 and the fourth plurality of information modules 226 may beblack. Furthermore, it should be appreciated that the first plurality ofinformation modules 218 and the third plurality of information modules224 and/or the second plurality of information modules 220 and thefourth plurality of information modules 226 may be various colors,levels of transparency or opacity, and/or levels of reflectivity orabsorptivity, or may have any other suitable characteristic allowing thebarcode to be readable by a reader. Then, the first 2D barcode iscompared to the second 2D barcode (block 450). For example, the first 2Dbarcode and the second 2D barcode may include binary data having valuesof zero (0) or one (1), which may correspond to each information modulebeing colored white for a binary value of zero (0) and black for abinary value of one (1). Then, the information modules are categorizedinto a first group 230, a second group 232, and a third group (block452). In an example embodiment, the first group 230 may include commoninformation modules between the first plurality of information modules218 of the first 2D barcode and the third plurality of informationmodules 224 of the second 2D barcode. Additionally, the second group 232may include unique information modules of the third plurality ofinformation modules 224 of the second 2D barcode. In an exampleembodiment, the third group may include unique information modules ofthe first plurality of information modules 218 of the first 2D barcode.For example, a computer may categorize all the black information modulesthat are common to the first 2D barcode and the second 2D barcode intothe first group 230. Additionally, the computer may categorize all theblack information modules that are unique to the first 2D barcode into athird group (i.e. the information modules that are black in the first 2Dbarcode, but white in the second 2D barcode). Also, the computer maycategorize all the black information modules that are unique to thesecond 2D barcode into a second group 232 (i.e., the information modulesthat were white in the first 2D barcode, but are black in the second 2Dbarcode). Then, the 2D barcode is printed using a static ink 194, afirst dynamic ink, and a second dynamic ink (block 454). In an exampleembodiment, the first group 230 may be printed in the static ink 194.Additionally, the second group 232 may be printed in the first dynamicink. The first dynamic ink may be adapted to activate in response to theoccurrence of a specific environmental factor. Furthermore, the thirdgroup may be printed in a second dynamic ink, and the second dynamic inkmay be adapted to activate in response to the occurrence of the specificenvironmental factor. For example, the first dynamic ink may be printedin white and active to black upon reaching 30° C. and the second dynamicink may be printed in black and activate to white upon reaching 30° C.It should be appreciated that the first and second dynamic ink may beprinted in several color combinations. In an example embodiment, thefirst dynamic ink and the second dynamic ink may be sensitive to anenvironmental factor such as temperature, time, time and temperature,freezing, radiation, toxic chemicals, or a combination of such factors,or the like. Additionally, in an example embodiment, the first dynamicink and the second dynamic ink may activate simultaneously. For example,the first dynamic ink and the second dynamic ink may both activate 72hours after printing such that the first dynamic ink changes from whiteto black and the second dynamic ink simultaneously changes from black towhite. Additionally, the first dynamic ink and the second dynamic inkmay both simultaneously activate after a temperature threshold is met(e.g., precision within 0.1° C. temperature range). Having the firstdynamic ink and second dynamic ink active simultaneously advantageouslyallows the 2D barcode to be readable at all times because the 2D barcodewill either be in a first state (i.e., the first 2D barcode) or a secondstate (i.e., the second 2D barcode).

FIG. 10A, FIG. 10B, and FIG. 10C are block diagrams of an example set ofinformation modules of a 2D barcode printed using process 440.Specifically, FIG. 10A shows the first plurality of information modules218 and the second plurality of information modules 220 of the first 2Dbarcode portion 216 (i.e., the set of dynamic data 136 is in the firststate 210). FIG. 10B shows the third plurality of information modules224 and the fourth plurality of information modules 226 of the second 2Dbarcode portion 222 (i.e., the set of dynamic data 136 is in the secondstate 212). FIG. 10C shows the first group 230, the third group 238, andthe second group 232 of information modules, from top to bottomrespectively. For example, the first group 230 includes commoninformation modules between the first plurality of information modules218 of the first 2D barcode portion 216 and the third plurality ofinformation modules 224 of the second 2D barcode portion 222, theseinformation modules are depicted as cross-hatched modules. The secondgroup 232 includes unique information modules of the third plurality ofinformation modules 224 of the second 2D barcode portion 222, this groupof information modules is depicted as a black module in the bottompicture in FIG. 10C. Additionally, the third group 238 includes uniqueinformation modules of the first plurality of information modules 218 ofthe first 2D barcode portion 216, and this group of information modulesis depicted by black modules in the middle picture of FIG. 10C. In thisexample, the first group 230 may be printed in a static ink 194, thesecond group 232 may be printed in a first dynamic ink 240, and thethird group 238 may be printed in a second dynamic ink 242. For example,the second group 232 may be printed with a first dynamic ink 240 adaptedto activate (i.e., transition from white to black) in response to theoccurrence of a specific environmental factor. Additionally, the thirdgroup 238 may be printed with a second dynamic ink 242 adapted toactivate (i.e., transition from black to white) in response to theoccurrence of the specific environmental factor. It should beappreciated that generating a 2D barcode with more than one dynamic inkadvantageously allows the 2D barcode to be printed with larger portionsof changing dynamic data that may include error detection and correctiondata 116. For example, by using multiple dynamic inks, the 2D barcode iscapable of changing several different regions of the 2D barcode enablinga non-privileged reader to read both the set of static data 134 and theset of dynamic data 136 in a plurality of states to give a plurality ofoutputs without the error detection and correction data 116 overwritinga designated output.

In an example, modules of the barcode may continuously (as contrastedwith step-wise) change color state in response to environmentalconditions. For example, a sensor-augmented two-dimensional barcodeincludes a substrate with two overlayers, a first layer and a secondlayer (e.g., sensor layer and barcode layer). The sensor layer and thebarcode layers may be printed in either order on top of the other on thesubstrate, so that either the sensor layer or the barcode layer may bethe first layer printed on the substrate, the other being the second(top) layer. In the barcode layer, a two-dimensional error-correctingbarcode symbol may be provided provided in a permanent light (nominally“white”) or dark (nominally “black”) color state. The barcode furtherincludes a plurality of white and black modules, the modules optionallybeing square, rectangular, or circular. The sensor layer comprises asensor dye may have a chemistry that is predictably responsive to aspecified environmental condition, undergoing continuous chemical orphysical state change between an initial state and an end state, causinga continuous color change in the color state of the sensor dye. Thesensor layer may optionally include one or more color calibrationpatches of known reflectivity to be used in autocalibration of thebarcode scanner at the reading color of interest. The calibrationpatches may be preprinted as part of the sensor layer and may appeareither adjacent to or at specific module positions within thesensor-augmented two-dimensional barcode.

As described above, both the sensory dye patch and the barcode may beprovided on the substrate. Depending on the order of printing, all orpart of a layer may be provided on a substrate without being in contactwith the substrate. For example, if a sensor dye patch is overprintedonto a barcode, portions of the sensor dye patch may not be in directcontact with the substrate. A sensor dye patch may be printed on asubstrate and a GS1 Data Matrix barcode may be overprinted on the sensordye patch. Additionally, a GS1 Data Matrix barcode may be printed on thesubstrate and the sensor dye patch may be printed over the barcode. Asillustrated in FIG. 32A and FIG. 32B, the barcode may be aligned withthe sensor dye patch 3210 before printing to ensure that the sensor dyepatch is properly positioned within the barcode (e.g., positioned withinthe invariant area). For example, the sensor dye patch may be positionedwithin the Invariant Area of the 2D barcode such that the upper leftcorner (ULC) of the Data Matrix may be aligned with the ULC of thepatch. It should be appreciated that the sensor dye patch 3210 can bepositioned in other regions within the 2D barcode. Additionally, thesensor dye patch 3210 may be positioned near the 2D barcode such that itis still associated with the barcode during scanning. The overprinted 2Dbarcode may include one or more Application Identifiers such as AI (01)for a GTIN-14 of the product, an AI (10) for lot number, an AI (17) forproduct expiration of the lot, AI (21) for a serial number of thetime-temperature label, and an AI that indicates the barcode includes atemperature exposure indicator. Additionally, an AI (90) or another partof the Data Matrix barcode data may identify or form the size and/orlocation of the temperature exposure indicator, such as the “+” shapedsensor dye patch 3210 within the barcode. An additional AI or othermethod of data encodation may be used in the 2D barcode havingparameters which describe the Arrhenius kinetics of the chemicalreaction equation of the color response to the sensor's specificenvironmental factor. After the Data Matrix barcode is overprinted onthe sensor dye patch 3210, the product is distributed through its normalsupply chain to an end user.

The time-temperature indicator (TTI) may be scanned at any point in thesupply chain (e.g., using a smartphone carrying a TTI reader App, orother special barcode reader) to ensure that the TTI labeled product hasnot yet expired. Once the 2D barcode is scanned, and the reflectivitypercentage of the sensor dye modules determined, the barcode readerdevice may determine the remaining labeled product life, expendedproduct life, or an expected expiration date. For example, given theremaining product life and the current temperature, the barcode readermay estimate the expiration date of the product if continuously storedat the reference temperature, and additionally if stored at anotherlower temperature.

An example method of reading of a sensor-augmented two-dimensionalbarcode symbol includes scanning and optically processing an image ofthe sensor-augmented two-dimensional barcode symbol using a barcodeimager or color camera using a pixel color identification system such aspreferably sRGB, including construction of a scanned pixel mapcontaining the preferably sRGB color value of the pixels in the scannedmodules of the sensor-augmented two-dimensional barcode symbol. The sRGBvalue of the pixels in the scanned modules contain color contributionsfrom both the barcode layer and the underlying or overprinted sensorlayer.

Optionally, the scanning process may include reading of one or morepreprinted color calibration patches containing adjacent to or withinthe sensor-augmented two-dimensional barcode symbol as part of thescanned pixel map. Optionally, these calibration patches may bepreprinted as part of the sensor layer and may appear either adjacent toor at specific module positions within the sensor-augmentedtwo-dimensional barcode.

The pixels of the modules in the scanned pixel map are then processedusing thresholding algorithm and/or voting algorithms to assigned abinary color value to each pixel, to form an equal-sized binarised pixelmap. The 2D barcode symbol may be identified from other graphicalobjects in the binarised pixel map and then identified 2D barcode may bedecoded to construct a symbol codeword sequence from the binarised pixelmap. In an example, the color value assignment may utilize the IEC61966-2-1:1999 standard RGB color space (sRGB).

Then, underlying data codewords may be recovered from the symbolcodeword sequence, preferably by utilizing error correction process onthe symbol codeword sequence. In an example, the error correctionprocess is Reed-Solomon Error Correction.

Processing the data codewords then determines the location, size, andproduct life equation parameters of the sensor dye region, andadditionally whether calibration patches are present, and if so theirrelative location and their reference reflectance value. The datacodewords are processed for identification of the pixels of the barcodemodules in the sensor dye region, and the average sRGB color isdetermined. Additionally, processing the sRGB color information mayoptionally include all or some of the steps of: capturing the incidentlight reflectance of pixels included in the sensor dye region, includingboth environmentally-sensitive pixels and color calibration patches;creating a colored digital light filter effect on reflectance data fromthe pixels to generate filtered colored image sRGB values; reducing thefiltered colored image values to greyscale values and creating agreyscale pixel map; correcting the relationship between greyscale andreflection percentage based on the color calibration patch values; anddetermining a reflectance percentage of the incident light at thescanning sample time. As used herein, a pixel map may be a map of binarybits (e.g., a bitmap), ternary bits, etc. For example a pixel map mayinclude greyscale values or RGB color values.

In order to obtain the product life data an image sensor such as asmartphone carrying a TTI App reader, may be used to scan the 2D barcodewith the embedded sensor dye patch 3210. For example, the image may becaptured from an image sensor, such as the smartphone camera, using aflash such as a smartphone white flash. The nominally white flashintensity may overwhelm the ambient light and set the color temperaturefor image capture by the sRGB sensor of the camera. The image sensor maycapture the white incident light reflectance of the pixels, includingthose modules (typically but not necessarily in the Invariant Area)containing the sensor dye. In lieu of the incident light being aspecified color a physical color filter may be positioned over thecamera lens when the sRGB image is captured. Alternatively, a digitalfilter may be applied over the sRGB image pixel map to creates a coloredlight filter effect on the reflectance data. As an example the digitalfilter may programmed to process the sRGB image based on an appropriatecenter wavelength and range, as in a bandpass filter. Then, the filteredcolor image RGB values may be reduced to a greyscale value (e.g., range0 to 255) and a greyscale pixel map for the Data Matrix barcode may becreated. In an example, the 2D barcode may include encoded data toprovide the appropriate inputs and may be used to program a barcodereader for reading color saturation and/or density of the sensor dyepatch 3210. For example, through encoded data and/or ApplicationIdentifiers (AIs), the barcode can automatically program the barcodereader to properly sense color reflectivity of the sensor dye patch3210. The encoded data in the Application Identifiers may also includethe appropriate lifetime equation parameters such that the reader canestimate the used and/or remaining product life at a specifiedtemperature for the scanned product.

The greyscale pixel map of the Data Matrix barcode may then beprocessed, e.g., using a standard Data Matrix procedure, such as ISO16022 Data Matrix with modifications such as replacing the ISO 15415Global Threshold algorithm with the Ultracode Color barcode Symbologydual-threshold ternary algorithm to the separate pixels into blackpixels, white pixels, and color pixels (i.e., pixels neither black norwhite). Once the color pixels are separated from the black and whitepixels, the remaining black pixels and white pixels may be processedaccording to the methods of ISO 16022. For example, this processingmethod identifies the square module positions and the module centers inthe Data Matrix pixel map using only the black and white pixels. The ISO16022 method then decodes the Data Matrix and recovers the GS1 AI data.

As discussed above, an Application Identifier may identify the size,shape and/or location of the sensor dye modules. Next, as an example, agroup of pixels in each identified sensor dye module in the InvariantArea may be sampled; for example, a 3×3 or larger number of pixels atthe center of each module may be sampled. As a widely-used example, eachpixel value may serve as a vote and the module is categorized as black,white, or color based on a majority vote. In a 3×3 sample, 2 pixels maybe voted as black, 6 pixels voted as color, and 1 pixel voted as white.Since 6 is a majority of the 9 total pixels, the module is categorizedas color, and the average sRGB color for that module determined byaveraging separately the R, G, B values of the 9 pixels voting. Theaverage sensor area sRGB value is formed by averaging separately the R,G, B values of all the identified dye sensor modules. Any color modulesfound other than the identified sensor modules may be ignored, andReed-Solomon Error Correction process will recover their underlying databit value.

It should be appreciated that by using error correction, a sensor dyepatch 3210 can advantageously be used within a 2D barcode, such as inthe Invariant Area, without affecting the readability of the overlying(or optionally underlying) 2D barcode. Through the use of errorcorrection, such as ISO 16022 Reed-Solomon Error Correction process,which corrects any erroneously identified modules, including the coloredmodules and unknown modules in the Invariant area, the underlying GS1Data Matrix is recovered. Thus, data from the underlying GS1 Data Matrixis advantageously processed in the standard manner without beingcorrupted by the continuously changing color of the sensor dye patch3210. Therefore, static product data can be read from the overprinted 2Dbarcode while dynamic product data, such as remaining product life,embedded within the barcode continuously changes due to environmentalexposure.

The remaining color modules (e.g., modules that have not beenoverprinted with black modules) are processed to determine the currentreflectance percentage R(t) at the time of barcode capture (t). Asillustrated in FIG. 33, the reflectance R(t) may be processed todetermine equivalent exposure time (t_(e)) at the reference temperature,preferably by using the Arrhenius equation. After determining theequivalent exposure time (t_(e)), the total expended product life todate and the remaining life at the reference temperature can beestimated. Product life may depend on exposure to both time andtemperature. For example, with constant exposure to 30° C., product lifemay be 2.5 weeks. However, with constant exposure to 38° C., productlife may only be 1 week, and therefore the cumulative time temperaturesensor (e.g., sensor dye patch 3210) advantageously indicates cumulativeexposure to environmental conditions, which is beneficial when previousstorage conditions in the supply chain are unknown. For example, timeand temperature conditions of a transportation truck may be unknown, buta cumulative time temperature sensor may advantageously capture andrepresent this exposure through its change in chemical or physical state(e.g., color state) to allow calculation of the remaining life of thedelivered product. Additionally, other calculations based on theequivalent exposure time (t_(e)) may be conducted the remaining life ofthe product if stored at a different temperature may also be estimated.

By encoding the 2D barcode with an AI containing parameters toautomatically program the image processing and provide product equationparameters for pre-stored algorithms, the barcode reader can determineproduct characteristics that are specific to each 2D barcode labeledproduct utilizing the same barcode reader. For example, use a reader,the remaining product life for a food product may be determined usingthe image sensor processing and equation parameters indicated by readingthe sensor enhanced 2D barcode on that food product; differentparameters may be stored in an AI in the 2D barcode on a vaccine vialand which the same barcode reader may read and use to program the imageprocessing and equation parameters to calculate the remaining life ofvaccine in the vial.

After scanning the barcode and obtaining the appropriate life data, thebarcode reader may display information about the marked product and theremaining life. For example, the barcode in FIG. 32A may represent afirst intermediate state 3220A, which is at some point after an initialstate (e.g., product has 100% remaining life), of a barcode on a medicalproduct (e.g., inactivated Polio vaccine) that may be scanned to revealproduct lifetime data, as illustrated in FIG. 34, such as a MonitorCategory: VVM7, 80 percent life remaining, expiration date (e.g.,calculated from the estimated remaining life or based on some othercriteria), and product authenticity. Additionally, the barcode data mayinclude AI (01) for a GTIN, AI(10) batch number, and AI (21) serialnumber that are displayed by the 2D barcode reader. The static barcodedata may contain product information such that the barcode reader isautomatically provided with the appropriate equation parameters andinputs to calculate the remaining product life according to the currentcolor state of the sensor dye patch 3210. For example, after reflectancedata is obtained from the image sensor, the appropriate equationparameters can be used to determine equivalent exposure time (t_(e)) atthe reference temperature, as illustrated in FIG. 33. Subtracting(t_(e)) from the lifetime gives the remaining product life at thereference temperature.

The sensor dye patch 3210 may transition between an initial state and anend state. FIG. 32B may represent a barcode (and sensor dye patch 3210)in a second intermediate state 3220B, after an initial state (e.g.,product has 100% remaining life) and the first intermediate state 3220Aillustrated in FIG. 32A. For example, the sensory dye patch may changecolor from clear to a solid color in the end state. To ensure accuracyof product life calculations, the 2D barcode may be encoded such that athreshold value of opacity in the sensor dye patch 3210 is identified asthe end of the product life. For example, the 2D barcode may be encodedsuch that the barcode reader determines that the sensor dye patchreaching 20 percent reflectance of the end state reflectance R(∞) isexpired, which may allow a reader to determine how long a product hasbeen expired for. For example, if the product expiration was set at thefinal reflectance percentage end state R(∞) of the sensor dye patch,then the rate of change of reflectance would be so slow it would beimpracticalto determine when the dye patch 3210 had reached the endstate. However, if the expiration is set at an intermediate state, whichis before the end state, the 2D barcode may be configured to alsoprovide information about cummulative environmental exposure afterexpiration. In another example, the end state may be used as thethreshold value. Similar to FIG. 32A, the barcode on the medical product(e.g., inactivated Polio vaccine) that may be scanned at some otherlevel of exposure to reveal product lifetime data. As illustrated inFIG. 35, the product has exceeded its suitable life and the barcodereader may display “Test Failed”. For example, due excessive exposure totime, temperature, or both time and temperature, the product may have noremaining life. Similar to FIG. 33, the static 2D barcode data carriesthe same GTIN, batch number, serial number, etc. and carries theparameters necessary to implement the R(t) equation shown in graphicalform in FIG. 33.

Additionally, the barcode data may be utilized to check on productauthenticity as an anti-counterfeiting measure. For example, all or partof the encoded AI (01) for a GTIN, AI(10) batch number, and AI (21) datamay used as validation data form this product instance. This validationdata is sent by the reader as a query to the manufacturer's database tosee if that validation data is associated there, meaning that theproduct carrying that barcode has already been registered. If thevalidation data is not matched in manufacturer's database, or is markedthere previously seen, already used or expired, then the authenticity ofthis product instance just scanned is questionable. A warning code maybe placed in the manufacturer's database that multiple instances of thesame barcode have been seen, to warn others who may receive anotheridentical product instance that at least one of the products instancesis counterfeit.

As discussed above, the remaining product life may be temperaturedependent. Therefore, at a new storage temperature different from thereference temperature, a new predicted expiration date may be calculatedby using barcode data and the scanner's currently measured equivalentexposure time (t_(e)) at the reference temperature.

FIG. 11 includes a flowchart of an example process 460 of generating a2D barcode. Although the process 460 is described with reference to theflowchart illustrated in FIG. 11, it will be appreciated that many othermethods of performing the acts associated with the process 460 may beused. For example, the order of many of the blocks may be changed, manyblocks may be intermittently repeated or continually performed, certainblocks may be combined with other blocks, and many of the blocksdescribed are optional or may only be contingently performed.

The example process 460 may begin with determining a set of payload data112 (block 462). In an example embodiment, the set of payload data 112may include a set of static data 134 and a set of dynamic data 136.Additionally, the set of dynamic data 136 may have a first state 210 anda second state 212. Next, a computer may generate a 2D barcode (block464). In an example embodiment, the 2D barcode may include an encodedversion of the set of static data, a dynamic region 192 which is adaptedto store an encoded version of the set of dynamic data, and errordetection and correction data 116. Then, a printer may print the 2Dbarcode using a static ink 194 and the encoded set of the dynamic dataon the dynamic region 192 using a dynamic ink 198 (block 466). In anexample embodiment, the 2D barcode may be attached to various productssuch as food product, pharmaceutical products, etc. In an exampleembodiment, the dynamic ink 198 may change states responsive to at leastone environmental change, such that the set of dynamic data 136 is ineither the first state 210 or the second state 212. Additionally, in anexample embodiment, the error detection and correction data 116 mayaccommodate for changes in the set of dynamic data 136 in the dynamicregion 192 such that the 2D barcode may be readable by a reader and mayproduce a first output when the set of dynamic data 136 is in the firststate 210 and the 2D barcode may be readable by a reader and may producea second output when the set of dynamic data 136 is in the second state212. In an example embodiment, the dynamic region 192 may be provided ina padding region. Additionally, the dynamic region 192 may be providedat an end of a data region. Providing a 2D barcode that includes errordetection and correction data 116 that accommodates for changes in theset of dynamic data 136 advantageously allows an individual to use anon-privileged reader and also advantageously allows an individual toobtain two different output readings using the non-privileged reader.For example, without generating a 2D barcode with error detection andcorrection data 116 that accommodates for changes in the set of dynamicdata 136 in the dynamic region 192, a non-privileged reader may onlyproduce the first output regardless of whether the 2D barcode included aset of dynamic data 136 in the first state 210 or a set of dynamic data136 in the second state 212.

FIG. 12 includes a flowchart of an example process 470 of reading a 2Dbarcode. Although the process 470 is described with reference to theflowchart illustrated in FIG. 12, it will be appreciated that many othermethods of performing the acts associated with the process 470 may beused. For example, the order of many of the blocks may be changed, manyblocks may be intermittently repeated or continually performed, certainblocks may be combined with other blocks, and many of the blocksdescribed are optional or may only be contingently performed.

The example process 470 may begin with a reader reading a set of staticdata 134 included in the 2D barcode (block 472). In an exampleembodiment, the 2D barcode may be printed in static ink 194 and dynamicink 198. Additionally, an encoded version of the set of static data maybe printed in the static ink 194. Next, a reader may read a set ofdynamic data 136 included in the 2D barcode (block 474). In an exampleembodiment, an encoded version of the set of dynamic data 196 may beprinted in the dynamic ink 198. Additionally, the set of dynamic data136 may be printed in a redundant space on the 2D barcode. Next, thereader may generate a first output of the set of static data 134 (block476). Then, the reader may generate a second output of the set ofdynamic data 136 (block 478). In an example embodiment, the secondoutput may depend on which state of the one of a plurality of states thedynamic data is in, such as greater than 30° C. or less than 30° C.

FIG. 13 includes a flowchart of an example process 490 of reading a 2Dbarcode. Although the process 490 is described with reference to theflowchart illustrated in FIG. 13, it will be appreciated that many othermethods of performing the acts associated with the process 490 may beused. For example, the order of many of the blocks may be changed, manyblocks may be intermittently repeated or continually performed, certainblocks may be combined with other blocks, and many of the blocksdescribed are optional or may only be contingently performed.

The example process 490 may begin with a reader reading a set of staticdata 134 included in the 2D barcode (block 492). In an exampleembodiment, the 2D barcode may be printed in static ink 194 and dynamicink 198. Additionally, an encoded version of the set of static data maybe printed in the static ink 194. Next, a reader may read a set ofdynamic data 136 included in the 2D barcode (block 494). In an exampleembodiment, an encoded version of the set of dynamic data 196 may beprinted in the dynamic ink 198. Additionally, the set of dynamic data136 may be printed in a dynamic region 192 of the 2D barcode. Next, thereader may generate an output of the set of static data 134 and the setof dynamic data 136 (block 496). In an example embodiment, the outputmay be a first output when the set of dynamic data 136 is in a firststate 210, and the output may be a second output when the set of dynamicdata 136 is in a second state 212.

FIG. 14 is a block diagram of a 2D barcode providing system. The systemmay include a computer 292 and a printer 290. The system may be used toprovide barcodes 102. The computer 292 may include one or more computerprograms or components. It will be appreciated that all of the disclosedmethods and procedures described herein can be implemented using one ormore computer programs or components. These components may be providedas a series of computer instructions on any conventional computerreadable medium or machine readable medium, including volatile ornon-volatile memory, such as RAM, ROM, flash memory, magnetic or opticaldisks, optical memory, or other storage media. The instructions may beprovided as software or firmware, and/or may be implemented in whole orin part in hardware components such as ASICs, FPGAs, DSPs or any othersimilar devices. The instructions may be configured to be executed byone or more processors, which when executing the series of computerinstructions, performs or facilitates the performance of all or part ofthe disclosed methods and procedures. Additionally, the computer 292 mayinclude a display and may have a connection to one or morecommunications channels such as the Internet or some other voice and/ordata network, including, but not limited to, any suitable wide areanetwork or local area network.

The computer 292 may include one or more processors electrically coupledby an address/data bus to one or more memory devices, other computercircuitry, and one or more interface circuits. The processor may be anysuitable processor, such as a microprocessor. The memory preferablyincludes volatile memory and non-volatile memory. Additionally, thememory may store a software program that interacts with the otherdevices in the barcode providing system. This program may be executed bythe processor in any suitable manner. The memory may also store digitaldata indicative of documents, files, programs, barcodes, etc. receivedfrom a computer or a barcode reader. Other computer circuitry mayinclude a wide variety of hardware components including ASICs, or otherspecialized circuitry for manipulating data in a specific format, suchas barcode data.

One or more displays, printers 290, and/or other output devices may alsobe connected to the computer 292 via interface circuits. The display maybe a liquid crystal display or any other type of display. The printer290 may print a barcode that is generated and received from the computer292. Additionally, one or more storage devices may also be connected tothe computer 292 via the interface circuits. For example, a hard drive,CD drive, DVD drive, and/or other storage devices may be connected tothe computer 292. The storage devices may store any type of data, suchas barcode data 100, image data, historical access or usage data, etc.

FIG. 15A and FIG. 15B is a block diagram of a 2D barcode reading system.The system may include a reader 200, the system may be used to readbarcodes 102. In an example embodiment, the reader 200 may be aprivileged reader or a non-privileged reader. The reader 200 may be adedicated barcode reader or an apparatus configured to read barcodessuch as a mobile device, a personal digital assistant or PDA, asmartphone, a laptop, a tablet computer, or a desktop computer, as wellas any other user devices. The reader 200 may be adapted to read 1D and2D barcodes, or may be adapted to read only 2D barcodes. The reader 200may also transmit, receive, or exchange data with other network devicesvia a communication network. A network device may be a computer 292, adifferent reader 200, or any other device accessible via a communicationnetwork. Also, certain data may be stored in a reader 200 which is alsostored on the server, either temporarily or permanently, for example inmemory or a storage device. The network connection may be any type ofnetwork connection, such as a cellular or wireless connection, anEthernet connection, digital subscriber line, telephone line, coaxialcable, etc. Access to a reader 200 or dynamic data 136 may be controlledby appropriate security software or security measures. An individualusers' access may be defined by reader 200 and limited to certain dataand/or actions. For example, a user may only have access to anon-privileged reader which may only be capable of reading the staticdata 134 on a barcode 102. Additionally, a user may have access to aprivileged reader which may be capable of reading just the dynamic dataor both the dynamic and static data on a barcode 102. Accordingly, usersand/or administrators of the barcode reading system may be required toregister with one or more readers 200. Additionally, various options formanaging data located within a reader 200 and/or in a server may beimplemented. For example, a management system may be implemented in thereader 200 and may update, store, and/or back up barcode data 100locally and/or remotely using any suitable method of data transmission.

The method of reading of a sensor-augmented two-dimensional barcodesymbol has a number of requirements. Reading a sensor-augmented DataMatrix symbol is possible

-   -   1. If only a limited number of modules change color state in the        sensor dye module pattern, and    -   2. If the changed modules are restricted to a small number of        utahs, and    -   3. If there is sufficient Reed Solomon Error Correction        capability in the underlying Data Matrix,

then the RSEC process may be utilized to recover the underlying codeworddata in the underlying Data Matrix prior to the color state changes ofmodules caused by sensor activation.

Table 5 shows the data and RSEC codeword capacities of all square DataMatrix symbols up to 26×26 and all rectangular Data Matrix symbols.

Each data codeword normally requires two RSEC codewords to recover theunderlying data. As an example, a 16×16 square Data Matrix which has acapacity for 12 data codewords (12 data utahs) and has 12 RSEC codewords(12 RSEC utahs). Thus if activated sensor modules change 4 data utahs inthat 16×16 symbol, then eight RSEC codewords are utilized to recover thedata in those for altered utahs. This leaves 4 additional RSEC codewordsavailable for correction of any other symbol damage.

TABLE 5 Total Data and RSEC Codewords for Different Sizes of Data MatrixSymbols Maximum Error Data Capacity Total Correction Symbol Size Al-Codewords Max % Size Rows Cols Digits phanum Bytes Data RSEC Cwds ECSQUARE SYMBOLS 10 × 10 10 10 6 3 1 3 5 2 63% 12 × 12 12 12 10 6 3 5 7 358% 14 × 14 14 14 16 10 6 8 10 5 56% 16 × 16 16 16 24 16 10 12 12 6 50%18 × 18 18 18 36 25 16 18 14 7 44% 20 × 20 20 20 44 31 20 22 18 9 45% 22× 22 22 22 60 43 28 30 20 10 40% 24 × 24 24 24 72 52 34 36 24 12 40% 26× 26 26 26 88 64 42 44 28 14 39% RECTANGULAR SYMBOLS  8 × 18 8 18 10 6 35 7 3 58%  8 × 32 8 32 20 13 8 10 11 5 52% 12 × 26 12 26 32 22 14 16 147 47% 12 × 36 12 36 44 31 18 22 18 9 45% 16 × 36 16 36 64 46 24 32 24 1243% 16 × 48 16 48 98 72 28 49 28 14 36%

Two types of reading processes for the sensor-augmented two-dimensionalbarcode are utilized, depending on whether or not the structure ofsensor dye module pattern has been encoded in the underlying DataMatrix. In the first case, where the the structure of sensor dye modulepattern has not been encoded in the underlying Data Matrix of thesensor-augmented two-dimensional barcode:

-   -   1. Scan and optically process the image as part of the Data        Matrix reading process and construct a scan binary pixel map        (e.g., bitmap) of the scanned image. See ISO/IEC 16022 for one        methodology.    -   2. Process the scan binary pixel map (e.g., bitmap) to construct        the underlying symbol codeword sequence,    -   3. Utilize the Reed-Solomon error correction process on the        symbol codeword sequence to recover the underlying data        codewords prior to any alteration by activated sensor modules.        See ISO/IEC 16022 for one methodology.    -   4. Construct an underlying binary pixel map (e.g., bitmap) from        the underlying data codeword sequence equal in size to the scan        binary pixel map (e.g., bitmap). See ISO/IEC 16022 for one        methodology.    -   5. At each bit position, Exclusive OR the scan binary pixel map        (e.g., bitmap) and the underlying binary pixel map (e.g.,        bitmap) to form a sensor digital information pixel map of the        same size as the scan binary pixel map.    -   6. Process the sensor digital information pixel map according to        the situational rules.

In the case where the the structure of sensor dye module pattern hasbeen encoded in the underlying Data Matrix of the sensor-augmentedtwo-dimensional barcode:

-   -   1. Scan and optically process the image as part of the Data        Matrix reading process and construct a scan binary pixel map of        the scanned image. See ISO/IEC 16022 for one methodology.    -   2. Process the scan binary pixel map to construct the underlying        symbol codeword sequence,    -   3. Utilize the Reed-Solomon error correction process on the        symbol codeword sequence to recover the underlying data        codewords prior to any alteration by activated sensor modules.        See ISO/IEC 16022 for one methodology.    -   4. Utilize the information encoded in the underlying data        codewords to determine within the scan binary pixel map a sensor        dye bit pattern containing the sensor digital information and        extract the binary information sequence in its appropriate bit        order.    -   5. When the sensor data has been BCH(15,5,7) encoded, use        standard methods the BCH error-correcting process to recover the        5-bit binary-encoded sensor data (or else decode decoding        fails).

Additionally, where a continuously changing sensor dye is used, forexample, to indicate cumulative exposure to an environmental condition,then a barcode reader may:

-   -   1. Process color information from a sensor dye region,        preferably by processing the binary pixel map to retrieve the        barcode data determining the location, size, and product life        equation parameters of the sensor dye region. Additionally, the        binary pixel map may be used to program the image sensor for a        particular sensor dye and product type.    -   2. Utilize the product life equation parameters to determine        remaining product life from the processed color information.

An embodiment utilizes a sensor dye that is configured to undergo acontinuous chemical or physical state change between an initial stateand an end state causing a change in the color state of the sensor dye.The current color state may indicate exposure (e.g., cumulativeexposure) to the environmental condition. For example, a sensor dyepatch, such as a cumulative time-temperature sensor may be embeddedwithin or associated within a 2D barcode. The cumulativetime-temperature sensor may continuously change color as the product isexposed to temperature over time.

An embodiment utilizes a temperature threshold-sensitive sensor dyechemistry. The sensor dye chemistry is W→X; white dye elementsoverprinted on black modules of the printed underlying Data Matrix on awhite printing media. It is assumed the sensor dye chemistry used isknown at the time of printing the Data Matrix symbol.

It is assumed that the data structure encoded in the Data Matrix symbolutilizes GS1 Application Identifiers (AIs) and conforms to the GS1General Specifications, V15 Issue 2 (January 2015)(http://www.gs1.org/docs/barcodes/GS1_General_Specifications.pdf) andlater.

Here 15-bit BCH(15,5,7) error correction used to encode 5 bits of sensordata. Since the overprinted sensor dye modules are white, the blackmodules corresponding to ‘1’ bits of the 15-bit BCH encoding needs to beprinted in underlying Data Matrix.

The sensor data BCH encoding along with 2 indicator bits to indicate thesensor dye chemistry in use utilizes only utahs 3, 5 and 6 of the 16×16ECC 200 Data Matrix symbol shown in FIG. 19 for exemplary purposes.Utahs 5, 6 and utah bits 3.6-3.8 are in the same bitmap positionrelative to the symbol ULC as in the invariant bitmap 410 in FIG. 19.

The indicator utah bits 3.6 and 3.7 indicate which of the sensor dyechemistries in Table 1 is in use. It is assumed that the sensor dyechemistry to be used is known at the time of printing of the DataMatrix.

Modules of the selected sensor chemistry overprint both utah bits 3.6and 3.7. Depending upon the sensor dye chemistry selected, bits 3.6 and3.7 will appear as in Table 4 when the sensor dye is in either theunactivated or activated state.

FIG. 20 shows the size 14 Data Matrix bitmap 505 from ISO/IEC 16022 andidentifies utah bit 3.8 and utahs 5 and 6 of the invariant bitmap 410.These are references 510, 520 and 530 respectively. The upper 5 bits ofutah 3, bits 3.1-3.5 referenced as 540, which are not in the invariantbitmap 410, may be used to encode additional information about thesensor dye chemistry in use and/or the sensor dye bit pattern.

In keeping with the GS1 System as defined in the GS1 GeneralSpecifications, the most widely used system for encoding information inData Matrix, the GS1 Application Identifier AI (90) may be used. AI (90)is reserved for information mutually agreed upon between tradingpartners, such as the presence of a sensor-enabled Data Matrix. Sincethe Application Identifiers may appear within any sequence in a GS1 DataMatrix, AI (90) will appear immediately after the FNC1 to ensure thatthe 15 BCH-encoded sensor bits B1-B15 are in the invariant bitmap 410.Since only 7-bit characters can be encoded in a GS1 applicationidentifier, the most significant bit 5.1 of utah 5 at 520 and bit 6.1 ofutah 6 at 530 and their color state both before and after activation areunimportant here.

As shown in FIG. 21, the invariant bitmap 410 portion of the 16×16 DataMatrix 600 includes 610 a, utah 1 bits 1.5 and 1.8; 620 utah 2; 630 autah 3 bits 3.6-3.8; 640 a utah 4 bits 4.3-4.8; 650 utah 5; 660 utah 6and 670 a utah 7 bits 7.2, 7.4, 7.5, 7.7 and 7.8. Note that other utahbits surrounding these seven utahs are shown in 600 for referencepurposes and easy correspondence with the bitmap of FIG. 20. Forexample, 610 b utah 1 bits 1.1-1.4, 1.6, and 1.7; 640 b utah 4 bits 4.1and 4.2; and 670 b utah 7 bits 7.1, 7.3, and 7.6.

The most significant bit B15 will be encoded in 630 a utah bit 3.8 at630 a. Utah 5 at 650 bits 5.2 through 5.8 will encode B14 through B7 ofthe BCH encoded sensor bits. Utah 6 at 660 bits 6.2 through 6.8 willencode bits B7 through B1 of the BCH encoded sensor bits.

Since a white to transparent sensor dye chemistry is utilized in thefirst preferred embodiment, the encoded sensor data bit black and whitepattern B15-B1 should be preprinted in the Data Matrix. The W→X sensordye is then overprinted on these encoded bits; her either all of thebits B1-15 or at least those of the black bits B15-B1 in the underlyingData Matrix.

Consider an example when the sensor data value is ‘4’. From Table 2, theBCH encoding B15-B1 is 001000111101011. Thus in the Data Matrix size 14bitmap in FIG. 20, utah bits 3.8, 5.2-5.8 and 6.2-6.8are set to encodethe black modules corresponding to these bits B15-B1, as these moduleswill be over printed with the white to transparent sensor dye. Utah bits5.1 and 6.1 are set to ‘0’ to ensure white modules are printed. Table 6shows the AI (90) data string is printed in the first seven utahs of the16×16 Data Matrix. Recall that each utah encodes one 8-bit codeword.

TABLE 6 AI (90) ASCII and Data Matrix Codeword String Example for W→XSensor Dye 7-bit ASCII Hexadecimal Utah AI (90) Data Input CodewordNotes 1 FNC1 Printer E8 FNC1 is not a 7-bit ASCII value, so a specialdependent character string is sent to the printer to include at seeNotes the start of the GS1 AI string 2 90 90 DC AI (90) 3 See Notes  536 Indicator bits 3.6, 3.7 = ”11”, B15 in 3.8 = ’0’ and Bits 3.1-3.4 setto “0011” to flag as an example of a W→X sensor dye chemistry product 4X X 59 Any 7-bit ASCII. Here ‘X’ ‘is used as an example of an additionaldata character 5 Example “ 23 Sets bits B14-B8 to print ‘0100011’ 6Example j 6B Sets bits B7-B1 to print ‘1101011’ 7 GS or FNC1 GS 1ETerminates AI (90) string

Bits 3.1-3.5 of utah 3 at 630 b are not in the invariant bitmap 410 asthey are on the bottom edge of the Data Matrix symbol for all sizes ofData Matrix. However data may be encoded here which is useful incommunicating specific information about the properties of the sensordye in use, and the sensor dye pattern which is encoded in the augmentedData Matrix.

Since only part 640 a of utah 4 appears in the invariant bitmap portion410, utah 4 is used as a spacer to ensure that bits B14-B1 are printedin utahs 5 and 6 at 650 and 660 respectively. Any 7-bit ASCII charactermay be encoded in utah 4. It is typically used for product-relatedinformation.

A 16×16 Data Matrix 700 printed only with the information in Table 6 isshown in FIG. 22. In FIG. 23, Data Matrix 800 is shown similar instructure to Data Matrix 600. Here however, the appropriate modules inthe invariant bitmap portion are set to either black or white as per theencodation underlying Data Matrix 700 in FIG. 22. The remaining DataMatrix codewords (utahs 8-12) are filled with pad characters to fill outthe 12 available data codewords. The last 12 codewords in the symbol(utahs 13-24) are RSEC error correction codewords. In FIG. 23, forconvenience the contents of utahs 8-24 are shown in grey, since they arenot relevant to the encoding of AI (90).

The visual image of the unactivated sensor-augmented Data Matrix 900 isshown in FIG. 24. Note the white sensor dye is overprinted indicatorbits 910. The overprinted BCH encoding 920 of B15-B1 of the sensor datais shown as white dye modules, indistinguishable from the unprinted andnot-overprinted white modules at utah bits 5.1 and 6.1. These have value‘00000000000000’, indicating the unactivated default sensor data value‘0’ for B15-B1 in this sensor dye chemistry system. Note indicator bitsare ‘00’ indicating the unactivated state of a W→X sensor dye chemistry.

Once activated, all sensor modules become transparent and the visualimage reverts to FIG. 22, showing the correct 15 bit BCH bit pattern001000111101011 which is decoded using one of the standard methods forBCH (15,5,7) described above to recover the sensor data value ‘4’.

The second preferred embodiment utilizes a temperature thresholdsensitive sensor dye chemistry X→B as is frequently represented by thethermally-activated leuco dye systems used to make thermal paper. Heretransparent dye modules are overprinted on white (unprinted) modules ofthe underlying Data Matrix printed on a white media.

Here also 15-bit BCH(15,5,7) error correction used to encode 5 bits ofsensor data which are also encoded by overprinting sensor dye modules onan underlying printed Data Matrix symbol only at bit map locations wherea black module is to appear upon sensor dye activation. It is alsoassumed here that the sensor dye chemistry used is known at the time ofprinting the Data Matrix symbol.

With any sensor dye chemistry that is initially transparent, differentprinters may be used to print the underlying Data Matrix symbol andlater overprint it with the sensor dye modules as a separate process,thus augmenting the Data Matrix information be known at time the sensormodules are overprinted. In an example embodiment, the two-dimensionalbarcode may be attached to various products such as food products,pharmaceutical products, biologics, or any other product that maybenefit from environmental, physical, or biological monitoring. Forexample, the bar code may be printed on or applied to a container forsuch a product.

The sensor data 15-bit BCH encoding pattern, along with 2 indicatorbits, is identical to that of the first preferred embodiment. From Table2, for an X→B sensor dye chemistry the printed indicator bits 3.6 and3.7 will be “10”. As in the first preferred embodiment, the upper 5 bitsof utah 3, bits 3.1-3.5 may be used to encode additional informationabout the sensor dye chemistry in use and/or the sensor dye bit pattern.

In keeping with the most broadly used system for encoding information ina Data Matrix, the GS1 Application Identifier AI (90) is used. AI (90)is reserved for information mutually agreed upon between tradingpartners, such as the presence of a sensor-enabled Data Matrix. AI (90)is used in utah 1-7 data structure format as in the first preferredembodiment, with the specific data here encoded in Table 7.

Consider the same example as in the first preferred embodiment when thesensor data value is 4. From Table 4, the BCH encoding B15-B1 is001000111101011. Since a transparent-to-black sensor dye chemistry isutilized in this second preferred embodiment, the BCH-encoded sensor dyemodules for what will become black modules in B15-B1 once active must beselectively overprinted on B15-B1 white modules in the underlying DataMatrix. The indicator bits 3.6 and 3.7 are also overprinted with sensordye modules.

Table 7 shows the AI (90) data string that is printed for this secondpreferred embodiment example in the first 7 utahs of the 16×16underlying Data Matrix 1000 in FIG. 25. The underlying Data Matrix 1000printed only with the information in Table 7 is shown in FIG. 25.

Referring to the ULC detail 1100 in FIG. 26, indicator bits 3.6 and 3.7are set to ‘1’ and ‘0’ respectively. Utah bits 3.8, 5.2-5.8 and 6.2-6.8are white modules corresponding to ‘0’ for all these bits B15-B1. Utahbits 5.1 and 6.1 are set to ‘1’ as they are not part of the BCH bitencoding sequence, and the Data Matrix standard ISO/IEC standardrequires every utah to have at least 1 black module. The remaining DataMatrix codewords (utahs 8-12) are filled with pad characters to fill outthe 12 available data codewords, which are specially encoded as perISO/IEC 16022. The last 12 codewords in the symbol (utahs 13-24) areRSEC error correction codewords. In FIG. 26, for convenience thecontents of utahs 8-24 are shown in grey, since they are not relevant tothe encoding of AI (90).

TABLE 7 AI (90) ASCII and Data Matrix Codeword String Example for X→BSensor Dye 7-bit ASCII Hexadecimal Utah AI (90) Data Input CodewordNotes 1 FNC1 Printer E8 FNC1 is not a 7-bit ASCII value, so a specialdependent character string is sent to the printer to include at seeNotes the start of the GS1 AI string 2 90 90 DC 3 See Notes C 44Indicator bits 3.6, 3.7 = ‘10’, B15 in 3.8 = ’0’ and Bits 3.1-3.4 set to“0100” to flag as an example of a W→X sensor dye chemistry product 4 H H49 Any 7-bit ASCII. Here ‘H’ is used as an example of an additional datacharacter 5 Example Printer 80 Sets bits B14-B8 to print ‘0000000’Dependent ‘~~’ is a common printer string as ~ is 7F and Data Matrixencodes ASCII value +1 6 Example Printer 80 Sets bits B7-B1 to print‘0000000’ Dependent See note for utah 5 7 GS or FNC1 GS 1E Terminates AI(90) string

The white modules of the underlying Data Matrix will be visible when thesensor-augmented Data Matrix barcode symbol is in the unactivated state,showing a BCH encoding for B15-B1 of 00000000000000, or sensor datavalue 0 as in the sensor-augmented Data Matrix 1000 in FIG. 25 and theData Matrix ULC detail 1100 in FIG. 26.

Once the sensor modules are activated, the visual sensor-augmented DataMatrix 1200 ideally will appear, as shown in in FIG. 27. The sensor databit pattern B15-B1 of 001000111101011 will now show, and through readingof the Data Matrix as described above using supplemental standard BCHdecoding techniques on the sensor dye bit pattern the sensor data value4 will be recovered. Note that the Data Matrix 1200 is the same as theunderlying Data Matrix 1000 with the overprinted sensor dyes modules nowactivated to black (but shown in dark grey as 1220 in FIG. 27, forclarity) to reveal the sensor data bit pattern B15-B1, and the indicatorbit pattern 1210 now as ‘11’.

The third preferred embodiment also utilizes a temperature-sensitivesensor dye chemistry. Here sensor dye modules in the unactivated colorstate are overprinted as a single sensor dye patch in the invariantbitmap of an underlying Data Matrix printed on a white media.

Any sensor dye chemistry R→S may be used, provided the arbitrary colorstates R and S both have 1) sufficient visual color state change and 2)sufficient contrast under 660 nm reader illumination that the R colorstate scans as W or B and the S color state scans as the complementarycolor, B or W. This enables the sensor dye patch, in either theunactivated or activated color state, to be both visually discriminableand machine readable to using the aforementioned Data Matrix readingtechniques to recover the sensor dye modules as an image and furtherprocess it to determine the activation state of the sensor dye patch.

Different printers, or different stations of the same printing press,may be used: First to print the underlying Data Matrix symbol and laterto overprint it with the sensor dye patch in a separate process.

In this example, an X→B sensor dye chemistry is illustrated. The sensordye patch is a square approximately 2×2 sensor dye modules in size. Itis framed by and centered in a white area 4×4 modules in size. This 4×4white frame is positioned in the invariant bitmap of a 16×16 DataMatrix, to ensure that the 4×4 white frame and its enclosed sensor dyepatch will always be in the same position relative to ULC over broadrange of Data Matrix symbol sizes.

FIG. 28 shows the position of the 4×4 white frame 1310 (from bit 3.6 tobit 5.8) within the example 16×16 Data Matrix symbol 1300. The 4×4 whiteframe 1310 contains 4 rows of utah bits comprised as follows:

-   -   Row 1: Bits 3.6, 3.7, 3.8, 4.3    -   Row 2: Bits 2.5, 5.1, 5.2, 5.3    -   Row 3: Bits 2.8, 5.3, 5.4, 5.5    -   Row 4: Bits 6.2, 5.6, 5.7, 5.8

Note that in this third preferred embodiment, bits 3.6 and 3.7 are notused as indicator bits, as in the first and second preferred embodimentsbut are rather here they are part of the 4×4 white frame.

A naive approach would be to create the underlying printed Data Matrixsymbol without data restriction and simply overprint white modules onthe 16 underlying Data Matrix modules in the 4×4 white frame 1310.Depending on the data encoded, and whether any black modules in 1310were deliberately overprinted, up to 5 utahs and their encoded codewordscould be deliberately damaged by overprinting. A conventional approachto the use of Reed-Solomon error correction to recover up to 5 damagedcodewords during reading a Data Matrix would require the use of up to 10of the 12 available RSEC codewords in a 16×16 Data Matrix symbol. Thiswould leave few RSEC codewords available for other accidental symboldamage.

As in the examples shown in the first and second preferred embodiments,in this example symbol data encoding is also performed using the GS1Application Identifier AI (90) and the utah 1-7 data structure format.Symbol data specific to the third preferred embodiment is encoded inTable 8. For convenience the contents of utahs 8-24 are shown in grey inData Matrix 1300, since they are not relevant to the encoding of AI(90).

Utahs 1 and 7 are unaffected by the creation of the 4×4 white frame1310, as they have no bits within 1310. Utah bits 2.5 and 2.8 are within1310. In the AI(90) data encoding bit 2.5=‘1’, and therefore utah 2 willbe deliberately damaged when overwritten by a white module. Utah 5 isentirely within the 4×4 white frame 1310. Since by the way data isencoded with ISO/IEC 16022 Data Matrix there will always be at least 1black module in any valid utah; therefore no matter what 7-bit ASCIIdata character is encoded in utah 5 it and its codeword will bedeliberately damaged by overprinting all utah 5 modules with whitemodules.

However, is possible to avoid deliberate damage to bit positions inutahs 3, 4 and 6 which are within the 4×4 white frame merely byrestricting the data allowed to be encoded in these codewords so thatthere are no black modules (1′ bits) in utah bits 3.6-3.8, 4.3, 4.6 and6.2. Assuming than ‘x’ represents a “don't care” bit position in eachspecific bit position within a utah, the allowable 8-bit Data Matrixcodeword forms are:

-   -   Utah 3:xxxxx000    -   Utah 4: xx0xx0xx    -   Utah 6: ×0xxxxxx

Table 8 shows the AI (90) data string for this 3^(rd) preferredembodiment example in the first 7 utahs of the 16×16 underlying DataMatrix 1400 in FIG. 29.

TABLE 8 AI (90) ASCII and Data Matrix Codeword String Example for Patchsensor 7-bit ASCII Hexadecimal Utah AI (90) Data Input Codeword Notes 1FNC1 Printer E8 FNC1 is not a 7-bit ASCII value, so a special dependentcharacter string is sent to the printer to include at see Notes thestart of the GS1 AI string 2 90 90 DC Data Matrix encodes 2digits/codeword 3 See Notes BEL 08 Binary form: xxxxx000 so bits 3.6-3.8= ‘000’. As an example, here bits 3.1-3.5 are set to “00001” to signal adye patch symbol with an X→B sensor dye chemistry 4 See Notes ‘Q’ 52Binary form: xx0xx0xx Here binary form 01010010 (7-bit ASCII ‘Q’) is anexample of an additional data character 5 Product ‘?’ 40 Any 7-bit ASCIIvalue. Here ASCII 63 Related corresponding to ‘?’ is used as an example.6 Product ‘7’ 38 Binary form: x0xxxxxx Related Any 7-bit ASCII valueless than 64 decimal may be used. As an example, ‘7’ (ASCII 55) is used.7 GS or FNC1 GS 1E Terminates AI (90) string

In FIG. 29, the underlying Data Matrix 1400 is printed only with theinformation in Table 8. The remaining Data Matrix data codewords (utahs8-12) are filled with Data Matrix pad characters. The last 12 codewordsin the symbol (utahs 13-24) are RSEC error correction codewords.

In FIG. 30, a 4×4 white frame 1510 is shown within the invariant bitmapof Data Matrix 1400, corresponding to the 4×4 white area 1310 in FIG.28. There are several ways of producing this 4×4 white frame 1510,including physically overprinting white modules on the underlying DataMatrix 1400. A better way is a modification to the Data Matrix encodingand symbol generation software for the underlying symbol is by alteringthe Data Matrix's encoded 14×14 bitmap to ensure that all bit positionsin 1510 are set to ‘0’ prior to conversion to black and white modules,or alternatively setting all modules to white in the 4×4 white area 1510prior to printing Data Matrix 1400. Thus, no black modules are everprinted in the 4×4 white area 1510 in the first place, requiring nowhite module overprinting step.

As in the second preferred embodiment, in the third preferred embodimenta secondary printing step is used to print an approximately 2×2 sensordye patch 1620 within the 4×4 white frame 1510, as shown in FIG. 31.Here the sensor dye patch 1602 is shown as activated (for clarity in apurple color).

One of the objectives of the third preferred embodiment that there is avisible change of the sensor dye patch 1520 color state within the 4×4white frame 1510 upon sensor dye activation. The second objective isthat a sensor dye chemistry is employed that has adequate contrast tothe Data Matrix reader as described above so that the sensor dye patch1620 is read as W or B modules when unactivated and read as thecomplementary color B or W modules when activated. Then the presence ofboth the unactivated and activated sensor dye patch can be machinereadable to a Data Matrix reader using reading methods described above.

A further improvement to the third preferred embodiment is in the DataMatrix reading process: It is to apply the knowledge that the codewordsof utahs 2 and 5 have been deliberately damaged to improve theefficiency of the Reed-Solomon error correction process in the symbolcodeword sequence recovery. Detection and correction of an erroneouscodeword at an unknown location in the combined data plus RSEC codewordsequence requires the use of 2 RSEC characters per damaged codeword.But, if the location of the damaged codewords are known prior toapplying the Reed-Solomon error correction process (in this case incodewords 2 and 5) then only 1 RSEC codeword is required to recover thecorrect codeword value of each identified damaged codeword. This leavesadditional unused RSEC codewords available for other accidental DataMatrix symbol damage correction.

Alternative embodiments include use of one or more of

-   -   other two-dimensional error correcting barcode symbols in place        of Data Matrix including QR Code, Aztec Code, MaxiCode, PDF417        and Dot Code;    -   alternate sensor dye chemistries utilizing color states other        than black, white or transparent, such as a continuously        variable color state between an initial state and an end state;    -   underlying printed two-dimensional error correcting barcode        symbols where either first color state or the second color state        of the symbol are colors other than black or white;    -   underlying two-dimensional error correcting barcode symbols        where either the first color state may be the unmarked media        surface and the second color state a directly-marked media        surface change, or vice versa;    -   overprinting a two-dimensional error correcting barcode symbol        on a color indicator or sensor dye patch that continuously        changes color;    -   overprinting a color indicator or sensor dye patch that        continuously changes color on a two-dimensional error correcting        barcode symbol.

In an exemplary aspect of the present disclosure, a sensor-augmentedtwo-dimensional barcode includes a substrate, a first layer provided onthe substrate, and a second layer provided on the substrate. The firstlayer includes a sensor dye region, which includes a sensor dye having achemistry that is configured, responsive to the occurrence of anenvironmental condition, to undergo a continuous chemical or physicalstate change between an initial state and an end state, causing a changein the color state of the sensor dye. The color state indicates exposureto the environmental condition. The second layer includes atwo-dimensional error-correcting barcode symbol, which includes aplurality of modules in a permanent color state. The modules areoptionally square, rectangular, or circular.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the second layer is aligned with a feature of the first layer,and wherein, preferably, the sensor dye region is positioned in aninvariant area of the two dimensional barcode such that an upper leftcorner of the second layer is aligned with an upper left corner of thefirst layer.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the environmental condition is selected from the groupconsisting of time, temperature, and time-temperature product, andwherein, preferably, the sensor dye continuously changes color statebetween the initial state and the end state when exposed to theenvironmental condition.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the second layer is printed over the first layer.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the first layer is printed over the second layer.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the second layer forms a readable barcode symbol in thesymbology of a two-dimensional barcode.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the two-dimensional error-correcting barcode symbol is from thesymbology group consisting of Data Matrix, QR Code, Aztec Code,MaxiCode, PDF417 and Dot Code symbologies.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the two-dimensional error-correcting barcode symbol utilizesReed-Solomon error correction.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the sensor dye is initially in a first color state whenunactivated and dynamically changes to a plurality of different colorstates within a range between the initial state and the end state beforereaching the end state.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the sensor dye reaches a predetermined intermediate statebetween the initial state and the end state when the specified conditionof the sensed property is beyond a threshold value, wherein thethreshold value is preferably a labeled product life.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the sensor dye dynamically changes to a plurality of differentcolor states related to one of expended product life and remaininglabeled product life.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the first layer optionally may provide sensor digitalinformation, the sensor digital information preferably encoded in aninvariant pixel map of the two-dimensional symbol, and more preferablyencoded as binary encoded sensor data, and more preferably the binaryencoded sensor data is in an error correcting code, preferably chosenfrom the group consisting of Hamming Codes, Bose-Chaudhuri-HocquenghemCodes, Golay Codes, Simplex Codes, Reed-Muller Codes, Fire Codes,Convolutional Codes, and Reed-Solomon Codes.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the two-dimensional error-correcting barcode includes encodedapplication identifiers, and wherein, preferably, a first applicationidentifier indicates the size and location of the sensor dye region anda second application identifier indicates product life equationparameters, wherein, preferably, the product life equation parametersare Arrhenius equation parameters.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, pharmaceutical, biological, or food product, preferably avaccine; a container holding the pharmaceutical, biological, or foodproduct, preferably a vaccine vial; and a sensor-augmentedtwo-dimensional barcode symbol provided on or in the container,preferably being applied to the outside surface of the container.

In an exemplary aspect of the present disclosure, method of reading of asensor-augmented two-dimensional barcode symbol includes opticallyscanning an image of the sensor-augmented two-dimensional barcode symbolto obtain color values for pixels in the image. Then, a scanned pixelmap containing the color values in the sensor-augmented two-dimensionalbarcode symbol is constructed, and the pixels in the scanned pixel mapare processed to assign a binary color value to each pixel and to form abinarised pixel map. The two-dimensional barcode symbol is identifiedand decoded in the binarised pixel map to recover a symbol codewordsequence. Next, underlying data codewords are recovered from the symbolcodeword sequence, preferably by utilizing error correction process onthe symbol codeword sequence. The data codewords are processed foridentification of barcode modules in a sensor dye region, and an averagecolor value of the barcode modules in the sensor dye region isdetermined. The average color value of the sensor dye region isprocessed to determine a reflectance percentage of incident light at atime of scanning.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the method includes processing sensor digital information pixelmap to identify the sensor dye region and determine from a color stateof the sensor dye region product life information.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, processing color information includes capturing white lightreflectance of pixels included in the sensor dye region, and creating acolored light filter effect on reflectance data from the scanned pixelsutilizing a filter to generate filtered colored image values of thepixels in a scanned pixel map.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the method further includes reducing the filtered colored imagevalues to greyscale values creating a greyscale pixel map.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, processing color information includes capturing white lightreflectance of pixels included in the sensor dye region, and creating acolored light filter effect on reflectance data from the scanned pixelsutilizing a filter to generate filtered colored image values of thepixels in a scanned pixel map.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, processing color information includes reducing the filteredcolored image values in the scanned pixel map to greyscale values,determining an average greyscale value of the average color value of thebarcode modules, and processing the average grey value to determine theincident light reflectance percentage at the time of scanning.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, processing the pixels in the scanned pixel map includesclassifying each pixel as one of a black pixel, a white pixel, and acolor pixel.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the black pixels, the white pixels, and the color pixels areused to form a ternarised pixel map, and the black and white pixels inthe ternarised pixel map are used to identify the two-dimensionalbarcode symbol in the ternarised pixel map.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the method includes classifying each pixel in the invariantarea as one of a black pixel, a white pixel, and a color pixel.Additionally, the method optionally includes determining the averagegrey value of the color pixels, and processing the average grey value todetermine the current reflectance percentage at the sampling time.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the sensor dye region is located in an invariant area of a DataMatrix barcode symbol.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the method further includes recovering a visual pattern orimage from the sensor digital information.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the sensor dye region includes a sensor dye that is responsiveto environmental factors including at least one of temperature, time,radiation, light, and toxic chemicals.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the sensor dye region includes a sensor dye that is responsiveto environmental factors including at least one of temperature, time,radiation, light, and toxic chemicals.

In an exemplary aspect of the present disclosure, a method of reading asensor-augmented two-dimensional barcode symbol includes opticallyscanning an image of the sensor-augmented two-dimensional barcode symbolto obtain a greyscale value for each pixel in the image. Then, agreyscale pixel map of the pixels in the sensor-augmentedtwo-dimensional barcode symbol is constructed. Next, the method includesprocessing the pixels in the greyscale pixel map to assign a binarycolor value to each pixel and to form a binarised pixel map.Additionally, identifying the two-dimensional barcode symbol in thebinarised pixel map is identified and decoded to recover a symbolcodeword sequence. Then, underlying data codewords from the symbolcodeword sequence are recovered utilizing an error-correction process onthe symbol codewords. The data codewords are processed foridentification of the barcode modules in a sensor dye region. Then, anaverage greyscale value of the barcode modules in the senor dye regionis determined, and the average greyscale values of the sensor dye regionare processed to determine a reflectance percentage of incident light ata time of scanning.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, a bandpass filter is utilized when optically scanning the imageto create the effect of monochrome illumination.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, a light source is used to optically scan the image, the lightsource being a monochrome light source, wherein preferably, themonochrome light source is a monochrome laser.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, a barcode imager is used to optically scan the image, andwherein the barcode imager is responsive only to greyscale values.

In an exemplary aspect of the present disclosure, a sensor-augmentedtwo-dimensional barcode includes a substrate; a two-dimensionalerror-correcting barcode symbol provided on the substrate; a first layerprovided on the substrate in a permanent color state; and a second layerprovided on the substrate. The bar code symbol further includes aplurality of modules, the modules optionally being square, rectangular,or circular, each module having one of a first color state or a secondcolor state. The second layer is optionally provided by overprinting thefirst layer, in a sensor dye module pattern, the sensor dye modulepattern containing sensor digital information. The second layer furtherincludes a sensor dye having a chemistry that is configured, responsiveto the occurrence of an environmental, physical or biological condition,to undergo a chemical or physical state change causing a change in thecolor state of the sensor dye, thereby changing the color state of asubset of the plurality of modules.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the environmental condition is selected from the groupconsisting of time, temperature, time-temperature product, light,humidity, gas vapor and nuclear radiation, and wherein, preferably, thesensor dye permanently changes color state when the environmentalcondition crosses a threshold value.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the first layer forms a readable barcode symbol in thesymbology of the two-dimensional barcode.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the two-dimensional error-correcting barcode symbol is from thesymbology group consisting of Data Matrix, QR Code, Aztec Code,MaxiCode, PDF417 and Dot Code symbologies.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the two-dimensional error-correcting barcode symbol utilizesReed-Solomon error correction.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the sensor dye is initially in a black, white or transparentcolor state when unactivated and changes to a different color state uponactivation.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the sensor dye permanently changes color state when thespecified condition of the sensed property is above or below a thresholdvalue.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the specified condition of the sensed property is the detectionof the presence of a biological organism, biological agent or biologicaltoxin, preferably by utilizing a colorimetric immunoassay.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the second layer provides sensor digital information, thesensor digital information preferably encoded in an invariant bitmap ofthe two-dimensional symbol, and more preferably encoded as binaryencoded sensor data, and more preferably the binary encoded sensor datais in an error correcting code, preferably chosen from the groupconsisting of Hamming Codes, Bose-Chaudhuri-Hocquenghem Codes, GolayCodes, Simplex Codes, Reed-Muller Codes, Fire Codes, ConvolutionalCodes, and Reed-Solomon Codes.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the sensor digital information encoded in the sensor dye modulepattern is a visual pattern or image.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, an article of manufacture includes pharmaceutical, biological,or food product, preferably a vaccine; a container holding thepharmaceutical, biological, or food product, preferably a vaccine vial;and a sensor-augmented two-dimensional barcode symbol provided on or inthe container, preferably being applied to the outside surface of thecontainer.

In accordance with another exemplary aspect of the present disclosure, amethod of reading of a sensor-augmented two-dimensional barcode symbolincludes scanning and optically processing an image of thesensor-augmented two-dimensional barcode symbol, including constructionof a scan binary bitmap from scanned modules of the sensor-augmentedtwo-dimensional barcode symbol. The method further includes constructinga symbol codeword sequence from the scan binary bitmap. Then, underlyingdata codewords are recovered from the symbol codeword sequence,preferably by utilizing error correction process on the symbol codewordsequence, the error correction process preferably being a Reed-Solomoncode. Next, the underlying data codewords are processed to form theunderlying symbol codeword sequence. The method further includesconstructing an underlying binary bitmap from the scan binary bitmapfrom the underlying symbol codeword sequence, the underlying binarybitmap preferably equal in size to the scan binary bitmap. At each bitposition an exclusive-OR of the scan binary bitmap and the underlyingbinary bitmap may be performed to form a sensor digital informationbitmap. Optionally, the method includes, processing the sensor digitalinformation bitmap to recover a binary information sequence thatincorporates binary encoded sensor data, preferably by processing thebinary information sequence as an error-correcting code sequence andutilizing an error correction process to recover binary-encoded sensordata. The error-correcting code preferably is selected from the groupconsisting of Hamming Codes, Bose-Chaudhuri-Hocquenghem Codes, GolayCodes, Simplex Codes, Reed-Muller Codes, Fire Codes, ConvolutionalCodes, and Reed-Solomon Codes.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the method includes processing the sensor digital informationbitmap to identify a sensor dye patch and determine from a color stateof the sensor dye patch whether or not activation of a sensor dye hasoccurred in response to an environmental condition.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the sensor dye patch is located in an invariant area of a DataMatrix barcode symbol.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the method includes recovering a visual pattern or image fromthe sensor digital information bitmap.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, an apparatus may perform a method of generating a 2D barcode,includes determining a set of payload data including a set of staticdata and a set of dynamic data, generating a 2D barcode which includesan encoded version of the set of static data and includes redundantspace, designating at least part of the redundant space as a dynamicregion which is adapted to store the set of dynamic data, and printingthe 2D barcode using a static ink and an encoded version of the set ofthe dynamic data on the dynamic region using a dynamic ink which changesstates responsive to at least one environmental change, such that theset of dynamic data is in one of a plurality of states. The set ofdynamic data is readable by a reader of the 2D barcode and the set ofstatic data is readable by the reader of the 2D barcode when the set ofdynamic data is in each one of the plurality of states.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with the preceding aspect, the dynamicink is responsive to environmental factors including at least one oftemperature, time, radiation, light, and toxic chemicals.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the dynamic ink is responsive to time and temperature.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the dynamic ink is responsive to freezing.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the dynamic ink permanently changes in response to anenvironmental factor.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the dynamic ink transitions from a first state to a secondstate in response to the occurrence of a specific environmental factorand returns to the first state when the specific environmental factor isno longer occurring.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the redundant space includes at least one of a plurality ofunused bits, a padding region, and an error detection and correctionregion.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, wherein the redundant space includes at least one of a formatinformation region, a version information region, and a reference dataregion.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, a non-privileged reader is capable of reading the static dataand cannot read the dynamic data of the 2D barcode.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, only privileged readers are capable of reading the static dataand cannot read the dynamic data of the 2D barcode.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, an apparatus may perform a method of providing a 2D barcode,includes determining a set of static data, determining a set of dynamicdata, generating a first 2D barcode, generating a second 2D barcode,comparing the first 2D barcode and the second 2D barcode andcategorizing the information modules into a first group and a secondgroup, and printing the 2D barcode in static ink and dynamic ink. Theset of dynamic data has a first state and a second state. The first 2Dbarcode includes an encoded version of the set of static data and theset of dynamic data in the first state. The set of static data and theset of dynamic data in the first state include a first plurality ofinformation modules and a second plurality of information modules. Thesecond 2D barcode includes an encoded version of the set of static dataand the set of dynamic data in the second state. The set of static dataand the set of dynamic data in the second state include a thirdplurality of information modules and a fourth plurality of informationmodules. The third plurality of information modules includes all of thefirst plurality of information modules plus a set of one or moreinformation modules. The second plurality of information modulesincludes all of the fourth plurality of information modules plus the setof one or more information modules. The first group includes commoninformation modules between the first plurality of information modulesof the first 2D barcode and the third plurality of information modulesof the second 2D barcode. The second group includes unique informationmodules of the third plurality of information modules of the second 2Dbarcode. The first group is printed in the static ink, and the secondgroup is printed in the dynamic ink. The dynamic ink is adapted toactivate in response to the occurrence of a specific environmentalfactor.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the first plurality of information modules and the thirdplurality of information modules are black modules and the secondplurality of information modules and the fourth plurality of informationmodules are white modules.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the first plurality of information modules and the thirdplurality of information modules are adapted to be visuallydistinguishable from a printing surface and the third plurality ofinformation modules and the fourth plurality of information modules arevisually indistinguishable from the printing surface.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, an apparatus may perform a method of providing a 2D barcode,includes determining a set of static data, determining a set of dynamicdata, generating a first 2D barcode, generating a second 2D barcode,comparing the first 2D barcode and the second 2D barcode andcategorizing the information modules into a first group, a second group,and a third group, and printing the 2D barcode using a static ink, afirst dynamic ink, and a second dynamic ink. The set of dynamic data hasa first state and a second state. The first 2D barcode includes anencoded version of the set of static data and the set of dynamic data inthe first state. The set of static data and the set of dynamic data inthe first state include a first plurality of information modules and asecond plurality of information modules. The second 2D barcode includesan encoded version of the set of static data and the set of dynamic datain the second state. The set of static data and the set of dynamic datain the second state include a third plurality of information modules anda fourth plurality of information modules. The first group includescommon information modules between the first plurality of informationmodules of the first 2D barcode and the third plurality of informationmodules of the second 2D barcode. The second group includes uniqueinformation modules of the third plurality of information modules of thesecond 2D barcode, and the third group includes unique informationmodules of the first plurality of information modules of the first 2Dbarcode. The first group is printed in the static ink, the second groupis printed in a first dynamic ink, and the third group is printed in asecond dynamic ink. The first dynamic ink is adapted to deactivate inresponse to the occurrence of a specific environmental factor, and thesecond dynamic ink is adapted to activate in response to the occurrenceof the specific environmental factor.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the first dynamic ink and the second dynamic ink are responsiveto environmental factors including at least one of temperature, time,radiation, light, and toxic chemicals.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the first dynamic ink and the second dynamic ink are responsiveto time and temperature.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the first dynamic ink and the second dynamic ink are responsiveto freezing.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the first dynamic ink and the second dynamic ink activatesimultaneously.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, an apparatus may perform a method of providing a 2D barcode,includes determining a set of payload data including a set of staticdata and a set of dynamic data having a first state and a second state,generating a 2D barcode which includes an encoded version of the set ofstatic data, a dynamic region which is adapted to store the set ofdynamic data, and error detection and correction data, and printing the2D barcode using a static ink and an encoded set of the dynamic data onthe dynamic region using a dynamic ink which changes states responsiveto at least one environmental change, such that the set of dynamic datais in the first state or the second state, and the error detection andcorrection data accommodates for changes in the set of dynamic data inthe dynamic region such that the 2D barcode is readable by a reader andproduces a first output when the set of dynamic data is in the firststate and the 2D barcode is readable by a reader and produces a secondoutput when the set of dynamic data is in the second state.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the dynamic region is provided in a padding region.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, the dynamic region is provided at an end of a data region.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, an apparatus may perform a method of reading a 2D barcodeincludes scanning a set of static data included in the 2D barcode,scanning a set of dynamic data included in the 2D barcode, generating afirst output of the set of static data, and generating a second outputof the set of dynamic data. The 2D barcode is printed in static ink anddynamic ink, and an encoded version of the set of static data is printedin the static ink. An encoded version of the set of dynamic data isprinted in the dynamic ink which changes states responsive to at leastone environmental change, such that the dynamic data is in one of theplurality of states. The set of dynamic data is printed in a redundantspace on the 2D barcode. The second output indicates on which state ofthe plurality of states the dynamic data is in.

In accordance with another exemplary aspect of the present disclosure,which may be used in combination with any one or more of the precedingaspects, an apparatus may perform a method of reading a two dimensional(2D) includes scanning a set of static data included in the 2D barcode,scanning a set of dynamic data included in the 2D barcode, andgenerating an output based on the set of static data and the set ofdynamic data. The 2D barcode includes static ink and dynamic ink. Anencoded version of the set of static data is printed in the static ink.An encoded version of the set of dynamic data is printed in the dynamicink, which changes states responsive to at least one environmentalchange, such that the dynamic data is in one of a plurality of states.The set of dynamic data is printed in the dynamic region. The output isa first output when the set of dynamic data is in a first state of theplurality of states, and the output is a second output when the set ofdynamic data is in a second state of the plurality of states.

It should be understood that various changes and modifications to theexample embodiments described herein will be apparent to those skilledin the art. Such changes and modifications can be made without departingfrom the spirit and scope of the present subject matter and withoutdiminishing its intended advantages. It is therefore intended that suchchanges and modifications be covered by the appended claims. Also, itshould be appreciated that the features of the dependent claims may beembodied in the systems, methods, and apparatus of each of theindependent claims.

Many modifications to and other embodiments of the invention set forthherein will come to mind to one skilled in the art to which theseinventions pertain, once having the benefit of the teachings in theforegoing descriptions and associated drawings. Therefore, it isunderstood that the inventions are not limited to the specificembodiments disclosed, and that modifications and other embodiments areintended to be included within the scope of the appended claims.Although specific terms are employed herein, they are used in a genericand descriptive sense only and not for purpose of limitation.

1. A sensor-augmented two-dimensional barcode, comprising: a substrate;a first layer provided on the substrate comprising a sensor dye region,the sensor dye region, the first layer further comprising a sensor dyehaving a chemistry that is configured, responsive to the occurrence ofan environmental condition, to undergo a continuous chemical or physicalstate change between an initial state and an end state, causing a changein the color state of the sensor dye, wherein the color state indicatesexposure to the environmental condition; and a second layer provided onthe substrate comprising a two-dimensional error-correcting barcodesymbol, the bar code symbol further comprising a plurality of modules ina permanent color state, the modules optionally being square,rectangular, or circular.
 2. The sensor-augmented two-dimensionalbarcode symbol of claim 1, wherein the second layer is aligned with afeature of the first layer, and wherein, preferably, the sensor dyeregion is positioned in an invariant area of the two-dimensional barcodesuch that an upper left corner of the second layer is aligned with anupper left corner of the first layer.
 3. The sensor-augmentedtwo-dimensional barcode symbol of claim 1, wherein the environmentalcondition is selected from the group consisting of time, temperature,and time-temperature product, and wherein, preferably, the sensor dyecontinuously changes color state between the initial state and the endstate when exposed to the environmental condition.
 4. Thesensor-augmented two-dimensional barcode of claim 1, wherein the secondlayer is printed over the first layer.
 5. The sensor-augmentedtwo-dimensional barcode symbol of claim 1, wherein the two-dimensionalerror-correcting barcode symbol is from the symbology group consistingof Data Matrix, QR Code, Aztec Code, MaxiCode, PDF417 and Dot Codesymbologies.
 6. The sensor-augmented two-dimensional barcode symbol ofclaim 1, wherein the two-dimensional error-correcting barcode symbolutilizes Reed-Solomon error correction.
 7. The sensor-augmentedtwo-dimensional barcode symbol of claim 1, wherein the sensor dye isinitially in a first color state when unactivated and dynamicallychanges to a plurality of different color states within a range betweenthe initial state and the end state before reaching the end state. 8.The sensor-augmented two-dimensional barcode symbol of claim 7, whereinthe sensor dye reaches an intermediate state between the initial stateand the end state when the specified condition of the sensed property isbeyond a threshold value, wherein the threshold value is preferably alabeled product life.
 9. The sensor-augmented two-dimensional barcodesymbol of claim 7, wherein the sensor dye dynamically changes to aplurality of different color states related to one of expended productlife and remaining labeled product life.
 10. The sensor-augmentedtwo-dimensional barcode symbol of claim 1, wherein the second layerprovides sensor digital information, the sensor digital informationpreferably encoded in an invariant pixel map of the two-dimensionalsymbol, and more preferably encoded as binary encoded sensor data, andmore preferably the binary encoded sensor data is in an error correctingcode, preferably chosen from the group consisting of Hamming Codes,Bose-Chaudhuri-Hocquenghem Codes, Golay Codes, Simplex Codes,Reed-Muller Codes, Fire Codes, Convolutional Codes, and Reed-SolomonCodes.
 11. The sensor-augmented two-dimensional barcode symbol of claim1, wherein the two-dimensional error-correcting barcode includes encodedapplication identifiers, and wherein, preferably, a first applicationidentifier indicates the size and location of the sensor dye region anda second application identifier indicates product life equationparameters, wherein, preferably, the product life equation parametersare Arrhenius equation parameters.
 12. An article of manufacture,comprising: pharmaceutical, biological, or food product, preferably avaccine; a container holding the pharmaceutical, biological, or foodproduct, preferably a vaccine vial; and a sensor-augmentedtwo-dimensional barcode symbol provided on or in the container,preferably being applied to the outside surface of the container.
 13. Amethod of reading of a sensor-augmented two-dimensional barcode symbolcomprising: optically scanning an image of the sensor-augmentedtwo-dimensional barcode symbol to obtain color values for pixels in theimage; constructing a scanned pixel map containing the color values inthe sensor-augmented two-dimensional barcode symbol; processing thepixels in the scanned pixel map to assign a binary color value to eachpixel and to form a binarised pixel map; identifying the two-dimensionalbarcode symbol in the binarised pixel map; decoding the identifiedtwo-dimensional barcode symbol in the binarised pixel map to recover asymbol codeword sequence; recovering underlying data codewords from thesymbol codeword sequence, preferably by utilizing error correctionprocess on the symbol codeword sequence; processing the data codewordsfor identification of barcode modules in a sensor dye region;determining an average color value of the barcode modules in the sensordye region; and processing the average color value of the sensor dyeregion to determine a reflectance percentage of incident light at a timeof scanning.
 14. The method of claim 13, wherein processing colorinformation includes: capturing white light reflectance of pixelsincluded in the sensor dye region; and creating a colored light filtereffect on reflectance data from the scanned pixels utilizing a filter togenerate filtered colored image values of the pixels in a scanned pixelmap.
 15. The method of claim 15, wherein processing color informationfurther includes: reducing the filtered colored image values in thescanned pixel map to greyscale values; determining an average greyscalevalue of the average color value of the barcode modules; and processingthe average grey value to determine the incident light reflectancepercentage at the time of scanning.
 16. The method of claim 13, whereinprocessing the pixels in the scanned pixel map includes classifying eachpixel as one of a black pixel, a white pixel, and a color pixel.
 17. Themethod of claim 17, wherein the black pixels, the white pixels, and thecolor pixels are used to form a ternarised pixel map, and the black andwhite pixels in the ternarised pixel map are used to identify thetwo-dimensional barcode symbol in the ternarised pixel map.
 18. Themethod of claim 13, wherein the sensor dye region is located in aninvariant area of a Data Matrix barcode symbol.
 19. The method of claim13, wherein the sensor dye region includes a sensor dye that isresponsive to environmental factors including at least one oftemperature, time, radiation, light, and toxic chemicals.
 20. A methodof reading of a sensor-augmented two-dimensional barcode symbolcomprising: optically scanning an image of the sensor-augmentedtwo-dimensional barcode symbol to obtain a greyscale value for eachpixel in the image; constructing a greyscale pixel map of the pixels inthe sensor-augmented two-dimensional barcode symbol; processing thepixels in the greyscale pixel map to assign a binary color value to eachpixel and to form a binarised pixel map; identifying the two-dimensionalbarcode symbol in the binarised pixel map; decoding the identified 2Dbarcode symbol in the binarised pixel map to recover a symbol codewordsequence; recovering underlying data codewords from the symbol codewordsequence utilizing an error-correction process on the symbol codewords;processing the data codewords for identification of the barcode modulesin a sensor dye region; determining an average greyscale value of thebarcode modules in the sensor dye region; and processing the averagegreyscale of the sensor dye region to determine a reflectance percentageof incident light at a time of scanning.
 21. The method of claim 20,wherein a optical bandpass filter is used when scanning the image tocreate the effect of monochrome illumination.
 22. The method of claim20, wherein a light source is used to optically scan the image, thelight source being a monochrome light source, wherein preferably, themonochrome light source is a monochrome laser.
 23. The method of claim20, wherein a barcode imager is used to optically scan the image, andwherein the barcode imager is responsive only to greyscale values.